cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 00:
02 | So last time we learned about how So last time we learned about how
diffusion turn noise into data by diffusion turn noise into data by diffusion turn noise into data by
gradually adding noise and gra… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 00:
52 | by an SDE with this score function and
then we can just do score matching to then we can just do score matching to then we can just do score matching to
learn uh how to solve the uh the learn uh how t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 01:
41 | That's exactly correct. Okay. Why don't
we just do linear interpolation of the we just do linear interpolation of the we just do linear interpolation of the
two things, right? Um so exact how two thin… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 02:
26 | this trajectory in this way. Um yeah so
remember the same noise same data. remember the same noise same data. remember the same noise same data.
Then learning the transformation along Then learning th… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 03:
28 | progress. So like 0.25 time um times uh progress. So like 0.25 time um times uh
data minus noise, right? Uh so basically data minus noise, right? Uh so basically data minus noise, right? Uh so basical… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 04:
20 | this is literally what we need to learn, this is literally what we need to learn,
right? This is like literally the only right? This is like literally the only right? This is like literally the only
t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 05:
17 | Right? How do you progress through the
chain? You just compute the displacement chain? You just compute the displacement chain? You just compute the displacement
for x and then directly apply the for … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 06:
26 | full matching. Class is over. Hey,
no, not really. Okay, fine. Uh but no, not really. Okay, fine. Uh but no, not really. Okay, fine. Uh but
basically the the question that we're basically the the ques… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 07:
10 | people start to de develop this thing.
Okay. So basically back in 2018 there Okay. So basically back in 2018 there Okay. So basically back in 2018 there
this like pretty seminal paper um uh this like … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 08:
05 | can just solve this OD by integrating can just solve this OD by integrating
through uh the velocity and then you through uh the velocity and then you through uh the velocity and then you
you'll be abl… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 08:
55 | Anyone else I haven't talked to today?
Nobody remember what is normalizing Nobody remember what is normalizing Nobody remember what is normalizing
flow? >> You can reconstruct the process. >> You can … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 09:
50 | invertible transformation because invertible transformation because
everything is deterministic. everything is deterministic. everything is deterministic.
Okay, any question? Okay, any question? Okay,… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 11:
01 | no new mass uh is is going to get added
and no mass is going to disappear and no mass is going to disappear and no mass is going to disappear
getting dropped. So like translate it getting dropped. So … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 11:
50 | probability flux. So imagine a flux is
like the amount of flow the amount of like the amount of flow the amount of like the amount of flow the amount of
water uh per unit per time that's going water u… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 12:
55 | in a container. It's in like a contained in a container. It's in like a contained
tank. So, the probability mass never tank. So, the probability mass never tank. So, the probability mass never
change.… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 14:
00 | in math in this way. So this is like in math in this way. So this is like
basically like how much probability gets basically like how much probability gets basically like how much probability gets
cha… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 15:
12 | two assumptions.
>> Uh two assumptions. One is that >> Uh two assumptions. One is that >> Uh two assumptions. One is that
add up to one. The second one is that add up to one. The second one is that ad… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 16:
04 | thing that like flow out
>> as physically inclined. >> as physically inclined. >> as physically inclined.
>> It is okay. >> It is okay. >> It is okay.
>> I'm just a civil engineer. >> I'm just a civil… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 17:
08 | the divergence connection again the divergence connection again
>> yes I I think I missed a P here actually >> yes I I think I missed a P here actually >> yes I I think I missed a P here actually
by t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 18:
21 | that like so this is the correct that like so this is the correct
divergence. Sorry that that that one I divergence. Sorry that that that one I divergence. Sorry that that that one I
missed P. Um but … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 19:
13 | the log probability of of the log
density. Okay. And this OD is actually density. Okay. And this OD is actually density. Okay. And this OD is actually
equal to so the derivative of the total equal to … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 20:
52 | So exactly so so
how do you train your continuous how do you train your continuous how do you train your continuous
normalizing flow from here? What do we normalizing flow from here? What do we normal… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 22:
04 | we actually need to do numerical
integration right. But this is at integration right. But this is at integration right. But this is at
training time. So that just means that training time. So that jus… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 23:
07 | numerical integration, right? So it
still has a lot of steps. Can we do still has a lot of steps. Can we do still has a lot of steps. Can we do
something that doesn't require a lot of something that d… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 24:
11 | just need to make sure that the velocity
actually match with the ground truth actually match with the ground truth actually match with the ground truth
velocity. velocity. velocity.
So you just do L2 … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 25:
12 | changing every day be like actually I
think that the the storm is moving north think that the the storm is moving north think that the the storm is moving north
or maybe not. We don't know, right? So … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 25:
58 | Yeah. That's right. That's right. You
just go straight. just go straight. just go straight.
Yeah, exactly. Uh so basically this is Yeah, exactly. Uh so basically this is Yeah, exactly. Uh so basically… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 26:
57 | just start from some prior which we can just start from some prior which we can
choose. So it doesn't matter we can choose. So it doesn't matter we can choose. So it doesn't matter we can
choose the p… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 27:
43 | the the data point then you can get a
marginal marginal marginal
uh yeah and also uh just the boundary uh yeah and also uh just the boundary uh yeah and also uh just the boundary
condition essentially… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 29:
03 | probability path and also just like how probability path and also just like how
like and reweed by how likely in general like and reweed by how likely in general like and reweed by how likely in gener… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 30:
38 | this average is like averaging by like this average is like averaging by like
how likely is the data point given your how likely is the data point given your how likely is the data point given your
cu… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 31:
44 | regularization condition. This can can
can be done. Uh and then uh basically can be done. Uh and then uh basically can be done. Uh and then uh basically
you uh input your um because we know you uh inp… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 32:
46 | can construct can generate the marginal
property path which is what we want. property path which is what we want. property path which is what we want.
Oh, I saw so many confusion. Oh, I saw so many co… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 33:
50 | Think about that. Isn't that pretty Think about that. Isn't that pretty
cool? Okay. Why? Why? Why is that? Okay. cool? Okay. Why? Why? Why is that? Okay. cool? Okay. Why? Why? Why is that? Okay.
So, t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 34:
46 | Like basically you if you swap uh the Like basically you if you swap uh the
dx1 term inside so you swap the order of dx1 term inside so you swap the order of dx1 term inside so you swap the order of
t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 35:
50 | we're compressing the bullet.
>> It's literally the same. It's literally >> It's literally the same. It's literally >> It's literally the same. It's literally
the same. It's equivalent. It's not even … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 36:
53 | Well we first have the the the endpoint
distribution and because we're distribution and because we're distribution and because we're
transforming it straight to the point transforming it straight to t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 37:
52 | >> Yeah. No no this is what we chose. This
is a design choice. So that so this is a design choice. So that so this is a design choice. So that so this
thing is generalizable to any kind of thing is ge… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 39:
09 | we can directly sample from
>> therefor you >> like previous lectures like gausian >> like previous lectures like gausian
assumption is used like this closure on assumption is used like this closure o… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 40:
22 | we talked about stuff when we u when we
actually uh decrease the the velocity at actually uh decrease the the velocity at actually uh decrease the the velocity at
which we adding noise. which we addin… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 41:
26 | this particular path. Only on this this particular path. Only on this
particular x0 to x1 path. But it's not particular x0 to x1 path. But it's not particular x0 to x1 path. But it's not
in general. i… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 42:
24 | >> because if you look at the if you look >> because if you look at the if you look
at the this the the left hand side right at the this the the left hand side right at the this the the left hand side… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 43:
16 | literally the same conference and they
all got into the same conference. all got into the same conference. all got into the same conference.
So yeah, so the first group is a group So yeah, so the firs… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 44:
00 | but you may have this question, right? I
feel like some people are having this feel like some people are having this feel like some people are having this
question but they didn't ask. So like questio… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 44:
58 | do is now let's say you you after
training with the linear interpolation training with the linear interpolation training with the linear interpolation
you already got this coupling. So the you already… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 45:
50 | right? So you can map from one point in right? So you can map from one point in
P 0 to another point in P1 and then you P 0 to another point in P1 and then you P 0 to another point in P1 and then you
… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 46:
34 | because you don't know like basically if
you if you only do one step there can be you if you only do one step there can be you if you only do one step there can be
like a lot of um uh uh like like lik… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 47:
37 | >> The third image is the linear >> The third image is the linear
interpolation of the new coupling. interpolation of the new coupling. interpolation of the new coupling.
>> That what is that doing? >… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 48:
17 | connected this side to that side and connected this side to that side and
that side to this side. There's no that side to this side. There's no that side to this side. There's no
there's no this anymo… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 48:
56 | perfectly trained into this process,
>> then you do it again. Yeah. Yeah. So, >> then you do it again. Yeah. Yeah. So, >> then you do it again. Yeah. Yeah. So,
this is just you you get it this is just… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 49:
44 | well. you're just like kind of random
random walking a little bit. Whereas for random walking a little bit. Whereas for random walking a little bit. Whereas for
flow matching, you don't really do flow… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 50:
27 | question is is flow matching also the question is is flow matching also the
same thing because if you think about it same thing because if you think about it same thing because if you think about it
y… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 51:
12 | can write everything as if it is flow can write everything as if it is flow
matching as well and then you can matching as well and then you can matching as well and then you can
actually just recover … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 52:
03 | still need to take care of that part.
But it turns out that we actually do But it turns out that we actually do But it turns out that we actually do
know how to take care of that part. And know how to… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 52:
44 | OD, right? So this is the just um say we OD, right? So this is the just um say we
have this particular OD and it turns out have this particular OD and it turns out have this particular OD and it turns… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 53:
30 | change for the probability means that
the marginalss are all the same. So this the marginalss are all the same. So this the marginalss are all the same. So this
just means that now if you have your ju… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 54:
36 | >> That's a great question. Um so for >> That's a great question. Um so for
example, I don't know if you guys example, I don't know if you guys example, I don't know if you guys
familiar with the RL. … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 55:
40 | Use the use the corresponding PF OD of Use the use the corresponding PF OD of
the SDE. Right? So you just plug it in the SDE. Right? So you just plug it in the SDE. Right? So you just plug it in
and t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 57:
07 | >> A lot of uh uh lot of PD getting
converted to OD. Is it are we just doing converted to OD. Is it are we just doing converted to OD. Is it are we just doing
it because it's simpler or do we you it b… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 57:
53 | diversity but but yes so like
essentially um if you're sampling from a essentially um if you're sampling from a essentially um if you're sampling from a
SDE then it's going to be yeah because SDE then… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 59:
06 | sampling chain through your by by by sampling chain through your by by by
backward integrating your velocity and backward integrating your velocity and backward integrating your velocity and
then duri… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 60:
12 | you know just like you know you just add
something divide something you it's the something divide something you it's the something divide something you it's the
same thing. Okay. same thing. Okay. sam… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 61:
01 | next week we're gonna be able to see
like like like
uh different options to further uh different options to further uh different options to further
improve what we have learned. Um so for improve what… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 61:
41 | the generation faster? And we're going
to learn uh to do it with or without to learn uh to do it with or without to learn uh to do it with or without
training retraining of the model. And training ret… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 00:
03 | Good morning everyone. Good morning everyone.
Today we are starting with the next Today we are starting with the next Today we are starting with the next
lecture of our course which is lecture of our … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 01:
10 | simple which you could have got in the simple which you could have got in the
first lecture itself. first lecture itself. first lecture itself.
But I don't want to show you that route But I don't want… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 02:
06 | keep the mathematics as simple as
possible [snorts] and I have taken possible [snorts] and I have taken possible [snorts] and I have taken
inspiration from an uh MIT course to inspiration from an uh M… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 03:
01 | model is a representation of a phenomena
a neural network is an example of a a neural network is an example of a a neural network is an example of a
model or any mathematical function which model or a… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 04:
22 | we have looked at different techniques
to solve this broader problem. It to solve this broader problem. It to solve this broader problem. It
started out with variational started out with variational s… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 05:
34 | one of the most consequential techniques
in modern [snorts] in modern [snorts] in modern [snorts]
generative modeling. And the reason it generative modeling. And the reason it generative modeling. And… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 06:
33 | and it works exceptionally well. You can
see here they have this action expert see here they have this action expert see here they have this action expert
and this action expert is built [snorts] and … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 07:
26 | formula comes about which is very
simple. So flow modeling at the heart of simple. So flow modeling at the heart of simple. So flow modeling at the heart of
it is very simple and it has huge number it… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 08:
45 | maybe you're sitting on a table right maybe you're sitting on a table right
now or you have a table in front of you. now or you have a table in front of you. now or you have a table in front of you.
U… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 09:
51 | with time. So the magnets that we have
placed on the table, what if these placed on the table, what if these placed on the table, what if these
magnets are constantly moving? magnets are constantly mo… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 10:
59 | take at multiple time steps and hence
you will get a vector field which is you will get a vector field which is you will get a vector field which is
changing with time. So now we are discussing time d… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 12:
12 | depends on not just the position in
space but also it depends on the time space but also it depends on the time space but also it depends on the time
snapshot. So it is a timed dependent snapshot. So … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 13:
07 | timed dependent vector field.
So another way of defining this is So another way of defining this is So another way of defining this is
people usually write this t as a people usually write this t as a… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 14:
11 | So for example, if we take this example
at at the top at time t equal to0, this at at the top at time t equal to0, this at at the top at time t equal to0, this
is what my velocity field looks like. is… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 15:
16 | So now we are asking the question how do
the position of particles which are the position of particles which are the position of particles which are
inside the space change according to inside the spa… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 16:
24 | particle if I know the velocity vector
field. This is exactly what this formula field. This is exactly what this formula field. This is exactly what this formula
says. says. says.
So if I have a parti… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 17:
59 | of x0
okay so now this if I write as delta x okay so now this if I write as delta x okay so now this if I write as delta x
by delta t by delta t by delta t
is equal to mu of u of x0. is equal to mu of… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 19:
08 | box end up at time t equal to 10 let's box end up at time t equal to 10 let's
say. Now if you know the exact velocity say. Now if you know the exact velocity say. Now if you know the exact velocity
at… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 20:
02 | box follows the vector field and the box follows the vector field and the
question I'm trying to address is where question I'm trying to address is where question I'm trying to address is where
does t… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 21:
06 | they represent the vector field which is
time dependent. That is why the arrows time dependent. That is why the arrows time dependent. That is why the arrows
are changing with time. And if we know are… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 22:
21 | but it is not quite correct because at but it is not quite correct because at
the end of it we are doing an the end of it we are doing an the end of it we are doing an
approximation. approximation. ap… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 23:
38 | everywhere in a lot of papers. But flow everywhere in a lot of papers. But flow
means something very specific. In our means something very specific. In our means something very specific. In our
mind, … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 24:
34 | flow is a collection of trajectories
which evolve according to the velocity which evolve according to the velocity which evolve according to the velocity
field. We have just seen that if we know field… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 25:
30 | now let us look at one single snapshot
because we have we have restricted with because we have we have restricted with because we have we have restricted with
the screen in in front of us. Now you the… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 26:
21 | trajectories and we get a overall sense
of the flow. You can see that this this of the flow. You can see that this this of the flow. You can see that this this
is probably something which is swirling.… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 27:
09 | let's let's see how the grid deforms.
Now note that the grid is going to Now note that the grid is going to Now note that the grid is going to
deform in a way that the vector field deform in a way tha… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 28:
09 | according to a trajectory which is
governed by an ordinary differential governed by an ordinary differential governed by an ordinary differential
equation and if we collect all these equation and if w… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 29:
26 | understand how the grid deforms. Yeah.
So this this does work. So this this does work. So this this does work.
Okay. So you can actually change these Okay. So you can actually change these Okay. So yo… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 30:
34 | using the word vector field also. Second
is the trajectories which are calculated is the trajectories which are calculated is the trajectories which are calculated
from the velocity field using the si… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 31:
46 | It is only one dimensional minus theta x
and it is independent of time. So at and it is independent of time. So at and it is independent of time. So at
every time snapshot you take a camera every time… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 32:
52 | is dxt by dt is equal to minus theta of
xt. xt. xt.
Now let's see if this is indeed the Now let's see if this is indeed the Now let's see if this is indeed the
flow. flow.
If we substitute this in xt,… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 34:
06 | is itself completely zero. So the
position stays zero. But position stays zero. But position stays zero. But
let's look at yeah if x0 is any any any let's look at yeah if x0 is any any any let's look … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 35:
12 | example
x0 into e to minus theta t and we get the exact same plot. and we get the exact same plot.
So we can actually play this and see how So we can actually play this and see how So we can actually … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 36:
03 | field and how the flow is a collection
of the trajectories. of the trajectories. of the trajectories.
[snorts] Okay. Now uh once we are getting Okay. Now uh once we are getting
confident with the defi… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 37:
06 | We take small steps in the direction of
the velocity field and we see how the the velocity field and we see how the the velocity field and we see how the
particle trajectory changes with time. particl… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 37:
56 | long is your simulation carried on.
Okay. So we initialize time as t equal Okay. So we initialize time as t equal Okay. So we initialize time as t equal
to0. The step size which is delta t is 1 to0. T… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 38:
56 | position. That is all is mentioned in
the flowchart. So basically the flowchart. So basically the flowchart. So basically
if we have a particle and if we know the if we have a particle and if we know … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 39:
45 | For example, when we are using flow
models for robotics or protein models for robotics or protein models for robotics or protein
synthesis, synthesis, synthesis,
people calculate the flow which is the… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 40:
49 | Now we have defined the word flow in the Now we have defined the word flow in the
initial part of the lecture. We will initial part of the lecture. We will initial part of the lecture. We will
underst… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 41:
41 | So this is exactly what we are going to So this is exactly what we are going to
do in flow models. Also our goal is to do in flow models. Also our goal is to do in flow models. Also our goal is to
con… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 43:
05 | So we use the same ordinary differential So we use the same ordinary differential
equation dx by dt is equal to u. But equation dx by dt is equal to u. But equation dx by dt is equal to u. But
this ti… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 43:
58 | we have added in flow model is that the we have added in flow model is that the
collection of these particles represent collection of these particles represent collection of these particles represent
… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 44:
54 | evolves from t =0 to t = 1. evolves from t =0 to t = 1.
So this is sort of what we want. We want So this is sort of what we want. We want So this is sort of what we want. We want
to start from the ini… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 45:
57 | the start distribution which is a simple
gshian. It is uniformly distributed. gshian. It is uniformly distributed. gshian. It is uniformly distributed.
There are maybe around 2,000 particles There are… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 46:
47 | from the start distribution to the end
distribution. Now here there are a lot of options. We Now here there are a lot of options. We
can convert can convert can convert
we can change the end distribut… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 47:
42 | replaced by a neural network. replaced by a neural network.
So there is another word for this in So there is another word for this in So there is another word for this in
literature which is also call… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 48:
29 | model is being trained and the loss is
going down. What is the meaning of this going down. What is the meaning of this going down. What is the meaning of this
statement? The flow model is going to be … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 49:
25 | see the animation over here once this
code runs. Yeah. So let's see uh as expected we Yeah. So let's see uh as expected we
start with the initial gshian start with the initial gshian start with the in… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 50:
30 | models we had a number of encoders and a
number of decoders. number of decoders. number of decoders.
So they were discrete. Here something a So they were discrete. Here something a So they were discre… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 52:
03 | the particles who are compressing inside
here they are moving out and here they are moving out and here they are moving out and
slowly resembling that spiral shape. So slowly resembling that spiral sh… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 52:
56 | initial problem of converting the uh initial problem of converting the uh
gshian distribution to our actual gshian distribution to our actual gshian distribution to our actual
distribution. What we sa… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 53:
50 | So the differential equation is also
termed as a neural ordinary differential termed as a neural ordinary differential termed as a neural ordinary differential
equation. So the terms that I have equat… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 54:
43 | ends our first lecture and in the next ends our first lecture and in the next
lecture we will understand what is the lecture we will understand what is the lecture we will understand what is the
targe… |