cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 00:
04 | So let's begin. Previously last class uh So let's begin. Previously last class uh
we have talked about what is diffusion we have talked about what is diffusion we have talked about what is diffusion
m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 00:
57 | reserve sorry um all right so just a reserve sorry um all right so just a
reminder what is the forward process reminder what is the forward process reminder what is the forward process
basically ally … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 01:
44 | sums up together is another gausian and
you can just sum up a bunch of gausians. you can just sum up a bunch of gausians. you can just sum up a bunch of gausians.
So that's why in the forward process … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 02:
35 | is the diffusion for process. All right.
Now, uh how now that we uh we have the Now, uh how now that we uh we have the Now, uh how now that we uh we have the
forward process to train to construct forw… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 03:
24 | mean of the gausian uh and in addition
we can actually rewrite this mean this we can actually rewrite this mean this we can actually rewrite this mean this
prediction of mean into something that predi… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 04:
08 | last time there's some questions about
why should we do this reparameterization why should we do this reparameterization why should we do this reparameterization
well because doing this regression is … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 04:
54 | this uh before I do that does anyone
know in the audience what is normalizing know in the audience what is normalizing know in the audience what is normalizing
flow by any No way. Nobody know. No way.… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 05:
45 | to uh latent is the inverse of the
mapping from latent to data. So this is mapping from latent to data. So this is mapping from latent to data. So this is
sort of like normalizing flows. Um and sort o… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 06:
40 | if you have a auto reggressive models if you have a auto reggressive models
then you need to break things up by the then you need to break things up by the then you need to break things up by the
chai… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 07:
25 | And uh for normalizing flow just by
looking at this can anyone tell me what looking at this can anyone tell me what looking at this can anyone tell me what
potential problem it can it can have. Yes. Y… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 08:
23 | least. Um but then it as we have learned
last time it also has a problem of like last time it also has a problem of like last time it also has a problem of like
very unstable training and also it very… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 09:
24 | of images rather than actually having to of images rather than actually having to
comput the comput the comput the
>> yes that's right so you just need to >> yes that's right so you just need to >> ye… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 10:
13 | model so what you do is you sort of just
like follow the gra so say suppose you like follow the gra so say suppose you like follow the gra so say suppose you
have a gradient that gives you like u have… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 11:
00 | space right so basically what you can
imagine is that you can start from imagine is that you can start from imagine is that you can start from
anywhere on the data space so in this anywhere on the dat… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 11:
45 | respect to the data point of the log
likelihood so what is happening is that likelihood so what is happening is that likelihood so what is happening is that
like so and then again like what you can li… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 12:
31 | just like in diffusion right your source
distribution and your target distribution and your target distribution and your target
distribution should always be in the distribution should always be in th… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 13:
28 | gradient of uh of of P
X uh with respect to X. That's fine. But X uh with respect to X. That's fine. But X uh with respect to X. That's fine. But
later on we're going to see why because later on we're… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 14:
15 | is uh machine learning 101 right. So
very easy very easy uh uh uh very easy very easy uh uh uh very easy very easy uh uh uh
formulation. And this L2 here uh it has formulation. And this L2 here uh it … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 15:
08 | like auto reggressive models. So it can like auto reggressive models. So it can
sample or generate everything all at sample or generate everything all at sample or generate everything all at
once once… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 15:
47 | this L2 without this ground truth score.
Turns out you don't even need to use the Turns out you don't even need to use the Turns out you don't even need to use the
ground truth score in your loss grou… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 16:
35 | simplify uh the L2 the the original L2 simplify uh the L2 the the original L2
into two parts here. And then the first into two parts here. And then the first into two parts here. And then the first
pa… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 17:
27 | can have the gradient of uh log of p
with respect to x into you can break it with respect to x into you can break it with respect to x into you can break it
into this thing right uh and then these int… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 18:
13 | and then you can basically just get
something like this. So um yeah this is something like this. So um yeah this is something like this. So um yeah this is
a this is like we we probably wouldn't a thi… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 18:
58 | need to remember is that the second part
of the two things left. It's going to of the two things left. It's going to of the two things left. It's going to
come boils down to the negative of the come b… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 19:
41 | loss that we get, we don't really need a
ground true score. It's it's like ground true score. It's it's like ground true score. It's it's like
actually pretty pretty sick. So all you actually pretty p… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 20:
38 | function has the same dimensionality of function has the same dimensionality of
the of the data, right? Yeah. So, you the of the data, right? Yeah. So, you the of the data, right? Yeah. So, you
can ju… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 21:
37 | no one else. Do you have a Do you have a
Do you have a Do you have a Do you have a
>> Well, yes, but it's kind of related to >> Well, yes, but it's kind of related to >> Well, yes, but it's kind of re… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 22:
42 | Literally Gausian. Okay. So basically
what you do is uh and this is what we what you do is uh and this is what we what you do is uh and this is what we
call dnoising score matching. Why so get call dn… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 23:
25 | is you just need a sample data and then
perturb it a little bit and now it perturb it a little bit and now it perturb it a little bit and now it
becomes this like it's easier to becomes this like it's… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 24:
15 | literally this. Uh so yeah, for for literally this. Uh so yeah, for for
those of you who wonder why log density those of you who wonder why log density those of you who wonder why log density
easier b… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 25:
25 | truth it's a perturbed ground truth then truth it's a perturbed ground truth then
you may have a question but like let's you may have a question but like let's you may have a question but like let's
m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 26:
04 | anyway it's fine. Um uh so what you do
is you first draw draw a sample from is you first draw draw a sample from is you first draw draw a sample from
your data space using some easy to your data space… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 26:
50 | So this is what could have happened if
you do laundry dynamic essentially. So you do laundry dynamic essentially. So you do laundry dynamic essentially. So
at the beginning, right, we just like at the… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 27:
44 | anyway you just train your model to
predict the score and then you use predict the score and then you use predict the score and then you use
dynamic to sample. Uh however does it dynamic to sample. Uh… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 29:
00 | survey. Let's survey. Uh we are starting
from this column again. from this column again. from this column again.
Hello friends. Uh Hello friends. Uh Hello friends. Uh
does anyone want to share their t… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 30:
06 | haven't been that I haven't been talked haven't been that I haven't been talked
to today. There's so many people in the to today. There's so many people in the to today. There's so many people in the
… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 31:
12 | is what we call the manifold
hypothesis. What does it mean is that hypothesis. What does it mean is that hypothesis. What does it mean is that
basically usually the data that we want basically usually… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 31:
59 | what here says like some area of the what here says like some area of the
data space will not have support. What data space will not have support. What data space will not have support. What
it means … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 32:
43 | manifold from some point from on the
manifold to some point that's not on the manifold to some point that's not on the manifold to some point that's not on the
manifold the score is going to blow up m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 33:
28 | then the second thing. Yeah. Yeah. Yeah.
>> What is this the left plot even showing >> What is this the left plot even showing >> What is this the left plot even showing
either? Oh sorry the the the s… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 34:
10 | space or entire data space now has
support uh and then the score matching support uh and then the score matching support uh and then the score matching
actually behave really really nicely actually be… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 34:
57 | image is eventually boiled down to say
10 features or something in that case 10 features or something in that case 10 features or something in that case
the manifold is load eventually how is the mani… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 35:
45 | there's no probability and that's why there's no probability and that's why
the score is going to blow up and adding the score is going to blow up and adding the score is going to blow up and adding
s… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 36:
42 | is like they they they train on the is like they they they train on the
clean data. So it's like not I mean yes clean data. So it's like not I mean yes clean data. So it's like not I mean yes
obviousl… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 37:
28 | without any data right? So what's going
to happen here is that um so say this is to happen here is that um so say this is to happen here is that um so say this is
like your actual data score that your… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 38:
14 | really know where to move in the low
density region. So you're just going to density region. So you're just going to density region. So you're just going to
move sort of randomly and then you you move… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 39:
02 | high density from higher density right high density from higher density right
so like say this is your like intended so like say this is your like intended so like say this is your like intended
data … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 39:
48 | >> it it depends on like what exactly is >> it it depends on like what exactly is
this distribution say if this uh this distribution say if this uh this distribution say if this uh
distribution is a m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 40:
34 | Okay.
All right. Any other question? Cool. Cool.
All right. So, we have all these All right. So, we have all these All right. So, we have all these
problems. problems. problems.
Is it problem? Is it p… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 41:
48 | thing doesn't exist anymore, right? And thing doesn't exist anymore, right? And
then now the perturbed like say you have then now the perturbed like say you have then now the perturbed like say you ha… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 42:
37 | large of a of a noise, then it you're
just learning a very noisy distribution. just learning a very noisy distribution. just learning a very noisy distribution.
That's not what we want. So how much Th… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 43:
33 | about which region that we are uh that about which region that we are uh that
we're s we should be sampling at, then we're s we should be sampling at, then we're s we should be sampling at, then
we're… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 44:
19 | sample from, you are going to just
gradually decrease the noise or the gradually decrease the noise or the gradually decrease the noise or the
exploration term that you're adding at exploration term t… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 45:
05 | have that next? uh we have the pseudo
code and you know the long dynam dynamic code and you know the long dynam dynamic code and you know the long dynam dynamic
left over. Yeah. But basically literall… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 45:
50 | And then at training time you do this
multi-level den noising score matching multi-level den noising score matching multi-level den noising score matching
because at each time you basically because at… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 47:
23 | Yeah, just just think about it. Uh,
basically the difference between DDPM, basically the difference between DDPM, basically the difference between DDPM,
which is what we learned last last time, which … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 48:
48 | that the the the score is sort of it has that the the the score is sort of it has
like another factor or it has another like another factor or it has another like another factor or it has another
coef… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 49:
49 | we have infinite numbers of noise level? we have infinite numbers of noise level?
What do you what do we think is going to What do you what do we think is going to What do you what do we think is goin… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 50:
47 | tailor estimation of of of the real
thing and you can actually write both of thing and you can actually write both of thing and you can actually write both of
them into this this format and uh if you … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 51:
42 | because you see how like basically if
you do xt plus delta t minus xt is going you do xt plus delta t minus xt is going you do xt plus delta t minus xt is going
to be this thing right. So it's going t… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 52:
28 | you just add some stoasticity. So this you just add some stoasticity. So this
is why it's called stochastic is why it's called stochastic is why it's called stochastic
differential equation. different… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 53:
28 | and we choose the for process right so
we know f and g uh and then it's we know f and g uh and then it's we know f and g uh and then it's
literally just gausian right so who literally just gausian rig… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 54:
21 | to use a oiler solver which is like to use a oiler solver which is like
probably simple simplest solver uh that probably simple simplest solver uh that probably simple simplest solver uh that
then you… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 55:
09 | apply the change in x to x and then we apply the change in x to x and then we
apply delta t to t apply delta t to t apply delta t to t
and that's it. and that's it. and that's it.
And then we iterate … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 56:
13 | written as this like this function this written as this like this function this
this uh so there are four process can be this uh so there are four process can be this uh so there are four process can … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 57:
20 | the G as variance and in the VE
formulation we do not have any formulation we do not have any formulation we do not have any
adjustment of the X like whatever X you adjustment of the X like whatever X… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 58:
09 | did diffusion. This class we did did diffusion. This class we did
scorebased model and we also learned scorebased model and we also learned scorebased model and we also learned
that they're basically … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 00:
05 | Hello everyone and welcome to the next Hello everyone and welcome to the next
lecture of the course principles of lecture of the course principles of lecture of the course principles of
diffusion mode… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 01:
09 | looked at other lectures. This is looked at other lectures. This is
supposed to be a standalone lecture to supposed to be a standalone lecture to supposed to be a standalone lecture to
help you get in… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 02:
06 | are going to discuss today. are going to discuss today.
The good thing is that this unified The good thing is that this unified The good thing is that this unified
approach has a lot of connection wit… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 03:
13 | Okay. So what do we know about
diffusion? In this image diffusion? In this image diffusion? In this image
I have shown a dye diffusing or I have shown a dye diffusing or I have shown a dye diffusing o… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 04:
37 | Can we or in other words can we recover Can we or in other words can we recover
the original structure? the original structure? the original structure?
So imagine you're sitting and you see So imagine… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 05:
29 | shape of the die.
But that is not the case. [snorts] But that is not the case. [snorts] But that is not the case. [snorts]
So reversing time is something which is So reversing time is something which … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 06:
23 | unpredictable. It's it's very random.
It's not moving in a specific direction, It's not moving in a specific direction, It's not moving in a specific direction,
but but but
they're moving up and down.… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 07:
30 | So when the dye was diffusing and when So when the dye was diffusing and when
we reversed the whole process, we reversed the whole process, we reversed the whole process,
can you tell this movement of… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 08:
29 | seconds have passed and someone tells me seconds have passed and someone tells me
that Rajett this is the particle which that Rajett this is the particle which that Rajett this is the particle which
i… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 09:
29 | physics and AI comes into the picture.
This is a well-known physical property This is a well-known physical property This is a well-known physical property
and it makes sense in when you when we and i… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 10:
33 | Okay. Now let us apply this concept Okay. Now let us apply this concept
to data. to data. to data.
Why are we applying this concept to Why are we applying this concept to Why are we applying this conc… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 11:
30 | of them have wide eyes.
And the question they ask us is, And the question they ask us is, And the question they ask us is,
can you find a distribution can you find a distribution can you find a distri… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 12:
35 | when you sample from it, you should get when you sample from it, you should get
something which lies within that something which lies within that something which lies within that
distribution. distrib… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 13:
28 | And to find such distribution we need an
algorithm which is clever enough which algorithm which is clever enough which algorithm which is clever enough which
can understand that okay uh cats have a ca… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 14:
24 | the diffusion process. Now this is where
it gets very interesting. In the next it gets very interesting. In the next it gets very interesting. In the next
set of slides I'm going to replace the set of… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 15:
37 | and then later we recover it.
So we destroy something only to recover So we destroy something only to recover So we destroy something only to recover
it after some point. it after some point. it after… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 16:
36 | visualize the dye being slowly filling visualize the dye being slowly filling
the liquid and the structure the liquid and the structure the liquid and the structure
disappearing disappearing disappear… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 17:
33 | do the opposite. In the reverse process,
we will start with noise and then slowly we will start with noise and then slowly we will start with noise and then slowly
we will move backwards and we will w… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 18:
33 | We have already said that we are going We have already said that we are going
to apply the diffusion process to data. to apply the diffusion process to data. to apply the diffusion process to data.
Bu… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 19:
41 | and you see this motion in a lot of
physical processes you see these motions physical processes you see these motions physical processes you see these motions
in polymers you see this even in the in p… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 20:
36 | On the right hand side we are zooming in
on specific particles. So here I have on specific particles. So here I have on specific particles. So here I have
taken taken taken
seven particles as referenc… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 21:
35 | So what am I exactly doing here? Well
imagine that instead of these tiny dots imagine that instead of these tiny dots imagine that instead of these tiny dots
you have 784 dots and all these dots are y… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 22:
35 | I'm planning to take all these images,
divide them into pixels and subject each divide them into pixels and subject each divide them into pixels and subject each
pixel to a brownian trajectory so that… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 23:
43 | two terms a drift term and a noise term. two terms a drift term and a noise term.
So DDPM has a specific formulation for So DDPM has a specific formulation for So DDPM has a specific formulation for
t… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 24:
32 | Well, the first thing I can see is that
I can obviously see some current lines I can obviously see some current lines I can obviously see some current lines
in the river. So I'll say that well in the … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 25:
24 | a steady drift term which is the current
of the lake or the river and the second of the lake or the river and the second of the lake or the river and the second
term is the diffusion term which are te… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 26:
20 | for this forward diffusion. for this forward diffusion.
Okay. So we have converted data to noise Okay. So we have converted data to noise Okay. So we have converted data to noise
and the change in the… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 27:
18 | You might relate this to a variational You might relate this to a variational
autoenccoder where there is an encoder autoenccoder where there is an encoder autoenccoder where there is an encoder
and t… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 28:
09 | with our discussion with the time
machine. We were asking that can we machine. We were asking that can we machine. We were asking that can we
reverse the reverse the reverse the
diffusion of the dye? … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 29:
05 | more than 40 years back in the year 1982
by a single author Brian Anderson. by a single author Brian Anderson. by a single author Brian Anderson.
And in this paper he wrote a formula And in this paper… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 30:
12 | appreciate all the researchers who spend appreciate all the researchers who spend
a lot of time in writing these papers. a lot of time in writing these papers. a lot of time in writing these papers.
O… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 30:
59 | of G of T which is the diffusion term in
the forward process. And the second term the forward process. And the second term the forward process. And the second term
is the score function. is the score … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 31:
52 | reversed from noise to the image and reversed from noise to the image and
this is where the learning will happen. this is where the learning will happen. this is where the learning will happen.
Why is… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 32:
44 | where the density of the sharks is
maximum. maximum. maximum.
Now score function is exactly like this Now score function is exactly like this Now score function is exactly like this
compass and sharks… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 33:
44 | density.
Now where does the gradient and log come Now where does the gradient and log come Now where does the gradient and log come
from? Well, the gradient comes from uh from? Well, the gradient come… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 34:
34 | to predict. Remember we started with a
discussion that we want to predict the discussion that we want to predict the discussion that we want to predict the
distribution of the cats and this distributi… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 35:
34 | For now let's simplify things. Let us For now let's simplify things. Let us
assume that the score function is assume that the score function is assume that the score function is
already learned. alrea… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 36:
30 | comprises of the drift in the forward comprises of the drift in the forward
diffusion and the factor which depends diffusion and the factor which depends diffusion and the factor which depends
on the … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 37:
26 | ask the question what is the variation ask the question what is the variation
in the values of these two points. It in the values of these two points. It in the values of these two points. It
turns ou… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 38:
15 | in terms of delta t. So if you have
delta t you calculate delta x then you delta t you calculate delta x then you delta t you calculate delta x then you
again substitute delta t calculate delta again … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 39:
04 | differential equation or an SDE.
This formulation was laid down in a This formulation was laid down in a This formulation was laid down in a
paper by Song and Arman in the year paper by Song and Arman… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 40:
02 | represents the noise term. So we go from
the first iteration to the second the first iteration to the second the first iteration to the second
iteration. Then we again do diffusion iteration. Then we … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 40:
54 | you because we know the exact equation
of the reverse process and we use that of the reverse process and we use that of the reverse process and we use that
same equation for sampling. same equation fo… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 41:
49 | data set consisting only of images of data set consisting only of images of
handwritten digits which are ones and handwritten digits which are ones and handwritten digits which are ones and
eights. ei… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 42:
46 | like these handwritten digits.
Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal
distribution. This is what where we want distribution. … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 43:
41 | in the image follows a brownie in
motion. motion.
Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing
in water our in water our in w… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 44:
40 | trajectory I have shown but in reality
we have these 784 pixels. So it will be we have these 784 pixels. So it will be we have these 784 pixels. So it will be
in a 784dimensional space which is in a 7… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 45:
31 | Now the next step is we know that the
microscopic brownian motion is microscopic brownian motion is microscopic brownian motion is
reversible. Uh because as you can see we reversible. Uh because as yo… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 46:
28 | as well.
And now you can see on the left hand And now you can see on the left hand And now you can see on the left hand
side we have noise and we are using the side we have noise and we are using the … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 47:
16 | diffusion term but this time the
trajectory is spread into two very trajectory is spread into two very trajectory is spread into two very
nicely and uh we do get a biodal nicely and uh we do get a bio… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 48:
08 | have seen this word score function in
many papers but only after going through many papers but only after going through many papers but only after going through
the SD framework we really understand t… |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 48:
59 | using a very simple approach which can using a very simple approach which can
be implemented in Google Collab and uh be implemented in Google Collab and uh be implemented in Google Collab and uh
this … |
diffusion-principles-vizuara |
L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models | 49:
50 | It's a very nicely written paper and
after this introduction you should be after this introduction you should be after this introduction you should be
able to understand it. Thank you very able to und… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 00:
04 | Hello everyone and welcome to this next Hello everyone and welcome to this next
lecture in of the course principles of lecture in of the course principles of lecture in of the course principles of
dif… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 01:
15 | of X.
And our objective then becomes to And our objective then becomes to And our objective then becomes to
determine determine determine
the probability distribution P5 of X the probability distribut… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 02:
30 | The objective is to match the output
image with the input image as close as image with the input image as close as image with the input image as close as
possible. possible. possible.
And here we look… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 03:
46 | intelligence can be used to produce intelligence can be used to produce
images which look like they belong to images which look like they belong to images which look like they belong to
the underlying… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 04:
42 | it becomes uniform.
We took these key ideas and we We took these key ideas and we We took these key ideas and we
constructed a diffusion model which is constructed a diffusion model which is construct… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 05:
49 | So you inject noise at every transition.
So this is the first transition, second, So this is the first transition, second, So this is the first transition, second,
third etc. third etc. third etc.
Now… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 06:
45 | So our objective is to minimize the mean So our objective is to minimize the mean
square error between epsilon and epsilon square error between epsilon and epsilon square error between epsilon and eps… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 07:
41 | something which is common in every
single image on earth. And at a first single image on earth. And at a first single image on earth. And at a first
glance you're not able to answer it glance you're n… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 08:
36 | in the field of image generation through
generative AI. Models were extremely generative AI. Models were extremely generative AI. Models were extremely
important in creating a new wave in the importan… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 09:
45 | models in a single cohesive framework. models in a single cohesive framework.
So this is something which is very So this is something which is very So this is something which is very
unique and and ve… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 10:
48 | So in the energy based models our So in the energy based models our
objective is to predict P5 of X using objective is to predict P5 of X using objective is to predict P5 of X using
energy functions. … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 11:
57 | drop this ball in this curve.
What will happen is that the ball will What will happen is that the ball will What will happen is that the ball will
slowly settle down, slowly fall down and slowly settl… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 12:
57 | down and settle on the ground? The
reason is that at on on on the ground reason is that at on on on the ground reason is that at on on on the ground
the potential energy of the apple is is the potenti… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 13:
55 | energy function can be a proxy for how energy function can be a proxy for how
probable that particular data point or probable that particular data point or probable that particular data point or
sampl… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 14:
52 | course draw this schematically and we
can say that okay we want this energy or can say that okay we want this energy or can say that okay we want this energy or
or this probability curve to follow or … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 15:
51 | my function should satisfy my function should satisfy
and people use an exponential function and people use an exponential function and people use an exponential function
to relate the energy to the p… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 17:
03 | density as p5 of x. density as p5 of x.
Then from the above graph we can find Then from the above graph we can find Then from the above graph we can find
out a function which represents the out a func… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 17:
56 | system u every system finally tries to
go in a position with the minimum go in a position with the minimum go in a position with the minimum
possible energy and this is something possible energy and t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 18:
46 | the whole idea behind energy based
models models
okay now let us discuss after this point okay now let us discuss after this point okay now let us discuss after this point
let's take an example suppos… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 20:
00 | add up to one? We need the probability
to add up to one because if we are to add up to one because if we are to add up to one because if we are
creating a lot of samples and we are creating a lot of s… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 20:
52 | so let's let's take that example again. so let's let's take that example again.
Uh let's say you have a dice. Okay. Uh Uh let's say you have a dice. Okay. Uh Uh let's say you have a dice. Okay. Uh
and… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 21:
42 | and the solution is that we can simply and the solution is that we can simply
normalize the probabilities by dividing normalize the probabilities by dividing normalize the probabilities by dividing
it… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 22:
48 | So for the discrete states it looked So for the discrete states it looked
like this summation of e ra to e5 of x like this summation of e ra to e5 of x like this summation of e ra to e5 of x
x goes fr… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 23:
46 | energy based models. In other words, how
do we find these energy functions which do we find these energy functions which do we find these energy functions which
correspond to each of these states? cor… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 24:
58 | push the good data up. push the good data up.
So I want to move this curve like this. So I want to move this curve like this. So I want to move this curve like this.
The left side I want to push down … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 25:
49 | higher probability. All of us understand higher probability. All of us understand
this. this. this.
Now we will again use the same Now we will again use the same Now we will again use the same
formula… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 26:
47 | Uh here E5 of X denotes the energy
function. Okay. So now if you break this down, Okay. So now if you break this down,
this is the first term that you get. uh this is the first term that you get. uh t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 28:
00 | so p5 of x can be written as integral of
p5 of x given z into p of z dz. p5 of x given z into p of z dz. p5 of x given z into p of z dz.
And there we said that this integral is And there we said that … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 29:
00 | bypasses the partition function.
So you might be thinking that what is So you might be thinking that what is So you might be thinking that what is
this new terminology which I've this new terminology … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 29:
56 | And because of that we come up with a
alternative objective function which alternative objective function which alternative objective function which
uses something called as the score uses something c… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 31:
05 | appreciation for people who actually sit appreciation for people who actually sit
down, they do the mathematics and they down, they do the mathematics and they down, they do the mathematics and they
c… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 32:
17 | to occur. to occur.
Let's let let's take a sample example. Let's let let's take a sample example. Let's let let's take a sample example.
Let us say these black dots these are Let us say these black do… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 33:
18 | score function does. It tells you the score function does. It tells you the
direction to where the data is located. direction to where the data is located. direction to where the data is located.
So i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 34:
09 | know where the data is where the score
is the maximum is the maximum is the maximum
we can start from any point and finally we can start from any point and finally we can start from any point and fina… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 34:
58 | trajectory?
Is is is it can I define it? Can I write Is is is it can I define it? Can I write Is is is it can I define it? Can I write
an equation which can tell me how do I an equation which can tell… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 35:
58 | Let's say I calculate the score for this Let's say I calculate the score for this
point. Where will this score point point. Where will this score point point. Where will this score point
towards to th… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 36:
53 | right or the left uh the value will
decline at x= to infinity or minus decline at x= to infinity or minus decline at x= to infinity or minus
infinity the value becomes zero. infinity the value becomes… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 37:
55 | Now this is just the first term. We have
to take a gradient of this also. to take a gradient of this also. to take a gradient of this also.
Gradient of a constant is zero and the Gradient of a constan… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 38:
44 | So uh this is what we get if we actually
plot the score function. You can see plot the score function. You can see plot the score function. You can see
that the arrows are pointed towards the that the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 39:
41 | score-based framework but score-based framework but
score-based methods score-based methods score-based methods
is something which has come before is something which has come before is something which… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 40:
40 | challenges which we started out with. We
wanted something which can uh help us wanted something which can uh help us wanted something which can uh help us
offer a tractable alternative to offer a trac… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 42:
09 | Now what what happens is that when you
take this gradient uh the the first term take this gradient uh the the first term take this gradient uh the the first term
is minus grad of e5 of x which is simp… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 43:
06 | that's all.
So the partition function becomes zero So the partition function becomes zero So the partition function becomes zero
and is not included in the calculation and is not included in the calcu… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 44:
02 | makes sure that the score of the
predicted model matches the score of the predicted model matches the score of the predicted model matches the score of the
true probability distribution. true probabil… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 45:
33 | So ideally we want
a loss which looks like this. Now the a loss which looks like this. Now the a loss which looks like this. Now the
question again becomes how do we question again becomes how do we q… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 46:
22 | Let us assume that we have done the
training. We have matched this score training. We have matched this score training. We have matched this score
with the uh score for the true model for with the uh … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 47:
09 | data point and I know where to go I can
navigate my way and I can find samples navigate my way and I can find samples navigate my way and I can find samples
from the distribution Okay. And uh this fro… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 48:
18 | Here I can visually see that this is the
deepest valley and I can see the deepest valley and I can see the deepest valley and I can see the
treasure is hidden there. treasure is hidden there. treasure… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 49:
10 | valley and I start here. and I want to
reach the bottom of the valley. So I'll reach the bottom of the valley. So I'll reach the bottom of the valley. So I'll
make a step like this. I'll again make a … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 50:
12 | would get your next location.
But what is the slope here? How do we But what is the slope here? How do we But what is the slope here? How do we
find the slope? This is my energy find the slope? This i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 51:
07 | exactly what we are doing here. We are exactly what we are doing here. We are
calculating the gradient of the energy calculating the gradient of the energy calculating the gradient of the energy
and u… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 51:
59 | this which is gradient of the energy
function. So the negative of this function. So the negative of this function. So the negative of this
actually is the score function itself. actually is the score … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 52:
52 | gradient is zero. So you are just
sitting there but there is another pit sitting there but there is another pit sitting there but there is another pit
which is even below which is even below which is … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 53:
40 | need to provide a shake which gives you
enough random energy to kick you out of enough random energy to kick you out of enough random energy to kick you out of
these small potholes so that you can the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 54:
29 | we are sampling from a gshian we are sampling from a gshian
distribution with a mean of xi and a distribution with a mean of xi and a distribution with a mean of xi and a
standard deviation of beta. s… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 55:
30 | the total variance of this xt + 1 is
maybe a constant or it's just one that maybe a constant or it's just one that maybe a constant or it's just one that
is my hypothesis is my hypothesis is my hypoth… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 56:
32 | Now we what we do is we replace this Now we what we do is we replace this
grad of energy by s of x which we have grad of energy by s of x which we have grad of energy by s of x which we have
already s… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 57:
16 | trajectory let's say I start from here
with this update I'll go like this with this update I'll go like this with this update I'll go like this
and then I'll finally I I I need to come and then I'll f… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 57:
59 | the score function which takes you from
one location to the another and finally one location to the another and finally one location to the another and finally
you end up at a space or a region where … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 58:
55 | x is known to us. It looks something
like this. The probability of finding like this. The probability of finding like this. The probability of finding
the samples is maximum at these two the samples i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 59:
55 | comments.
So first we define the setup which are So first we define the setup which are So first we define the setup which are
the two mountain setup as as we saw the two mountain setup as as we saw t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 60:
40 | noise. We use the exact same equation
which we discuss in theory which we discuss in theory which we discuss in theory
and then we visualize the path. and then we visualize the path. and then we visua… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 61:
28 | It's it's called a drunk hiker.
It's like someone is drunk and walking It's like someone is drunk and walking It's like someone is drunk and walking
here and there and finally reaching the here and th… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 62:
18 | So I want all of you to run this tweak a
little things here and there have fun little things here and there have fun little things here and there have fun
with the notebook and really develop with the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 63:
15 | since sampling with langu dynamics only
requires the score like we saw in this requires the score like we saw in this requires the score like we saw in this
formula it only requires the score we formu… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 64:
27 | and score matching fits this vector and score matching fits this vector
field by minimizing the mean square field by minimizing the mean square field by minimizing the mean square
error between the tr… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 65:
40 | know how to pronounce this name.
uh however they they came up with a uh however they they came up with a uh however they they came up with a
technique to technique to technique to
distill that objecti… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 66:
47 | a summation of this and a constant.
So basically minimizing this meant that So basically minimizing this meant that So basically minimizing this meant that
we want to minimize this and you see here we… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 68:
01 | like this. A1 A2 A3
A4 A5 A6 A4 A5 A6 A4 A5 A6
A7 A8 and A9. So the trace of this A7 A8 and A9. So the trace of this A7 A8 and A9. So the trace of this
matrix is simply the addition of a1 + a5 matrix … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 69:
11 | uh it is it is completely fine for our uh it is it is completely fine for our
purposes what we need to understand is purposes what we need to understand is purposes what we need to understand is
the i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 70:
05 | Which makes a lot of sense, right? In Which makes a lot of sense, right? In
the previous example that we looked at the previous example that we looked at the previous example that we looked at
these t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 71:
14 | where we are trying to make the data
samples appear as syncs. samples appear as syncs. samples appear as syncs.
which is what you see over here also. If which is what you see over here also. If which … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 72:
07 | more to this term.
So the this term drives So the this term drives So the this term drives
scores to zero in the higher probability scores to zero in the higher probability scores to zero in the highe… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 73:
01 | Now uh once we have understood this, Now uh once we have understood this,
this is where the actual training of the this is where the actual training of the this is where the actual training of the
uh … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 73:
57 | distribution. So in a way we will take a
look at the whole uh whole pipeline. look at the whole uh whole pipeline. look at the whole uh whole pipeline.
Okay, let's try to understand this. Okay, let's … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 75:
18 | loss is made up of two terms. The first loss is made up of two terms. The first
is the trace of the Jacobian is the trace of the Jacobian is the trace of the Jacobian
which is what we discussed before… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 76:
10 | predictions for the x score and the y
score to create a vector. score to create a vector. score to create a vector.
Next we have a training loop which is a Next we have a training loop which is a Next… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 77:
00 | completely appear like a sync but there completely appear like a sync but there
are arrows coming in from the left and are arrows coming in from the left and are arrows coming in from the left and
the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 77:
49 | we go to any other point. So we are we go to any other point. So we are
going to see trajectories like these. going to see trajectories like these. going to see trajectories like these.
And in this ex… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 78:
33 | perturve ourselves from the valleys so perturve ourselves from the valleys so
that we don't miss any other treasures that we don't miss any other treasures that we don't miss any other treasures
which… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 79:
19 | them yet. We will make the connection in
the next lecture. the next lecture. the next lecture.
So initially used to enable efficient So initially used to enable efficient So initially used to enable e… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 80:
15 | Scores are given by gradient of
logarithm of uh the probability. It logarithm of uh the probability. It logarithm of uh the probability. It
turns out that the scores appear as turns out that the score… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 81:
02 | neural network to match the scores and neural network to match the scores and
to do that we use a score matching loss to do that we use a score matching loss to do that we use a score matching loss
wh… |