cmu-10799-diffusion-flow |
L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching | 32:
04 | and G GMMN can also be seen as a special and G GMMN can also be seen as a special
case of Imm. Uh case of Imm. Uh case of Imm. Uh
you can easily see that when if you set you can easily see that when i… |
cmu-10799-diffusion-flow |
L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching | 32:
53 | essentially in this paper the drift
field for their uh for their model is field for their uh for their model is field for their uh for their model is
also parameterized via uh via MMD also parameteriz… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 00:
02 | So last time we learned about how So last time we learned about how
diffusion turn noise into data by diffusion turn noise into data by diffusion turn noise into data by
gradually adding noise and gra… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 00:
52 | by an SDE with this score function and
then we can just do score matching to then we can just do score matching to then we can just do score matching to
learn uh how to solve the uh the learn uh how t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 01:
41 | That's exactly correct. Okay. Why don't
we just do linear interpolation of the we just do linear interpolation of the we just do linear interpolation of the
two things, right? Um so exact how two thin… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 02:
26 | this trajectory in this way. Um yeah so
remember the same noise same data. remember the same noise same data. remember the same noise same data.
Then learning the transformation along Then learning th… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 03:
28 | progress. So like 0.25 time um times uh progress. So like 0.25 time um times uh
data minus noise, right? Uh so basically data minus noise, right? Uh so basically data minus noise, right? Uh so basical… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 04:
20 | this is literally what we need to learn, this is literally what we need to learn,
right? This is like literally the only right? This is like literally the only right? This is like literally the only
t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 05:
17 | Right? How do you progress through the
chain? You just compute the displacement chain? You just compute the displacement chain? You just compute the displacement
for x and then directly apply the for … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 06:
26 | full matching. Class is over. Hey,
no, not really. Okay, fine. Uh but no, not really. Okay, fine. Uh but no, not really. Okay, fine. Uh but
basically the the question that we're basically the the ques… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 07:
10 | people start to de develop this thing.
Okay. So basically back in 2018 there Okay. So basically back in 2018 there Okay. So basically back in 2018 there
this like pretty seminal paper um uh this like … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 08:
05 | can just solve this OD by integrating can just solve this OD by integrating
through uh the velocity and then you through uh the velocity and then you through uh the velocity and then you
you'll be abl… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 08:
55 | Anyone else I haven't talked to today?
Nobody remember what is normalizing Nobody remember what is normalizing Nobody remember what is normalizing
flow? >> You can reconstruct the process. >> You can … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 09:
50 | invertible transformation because invertible transformation because
everything is deterministic. everything is deterministic. everything is deterministic.
Okay, any question? Okay, any question? Okay,… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 11:
01 | no new mass uh is is going to get added
and no mass is going to disappear and no mass is going to disappear and no mass is going to disappear
getting dropped. So like translate it getting dropped. So … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 11:
50 | probability flux. So imagine a flux is
like the amount of flow the amount of like the amount of flow the amount of like the amount of flow the amount of
water uh per unit per time that's going water u… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 12:
55 | in a container. It's in like a contained in a container. It's in like a contained
tank. So, the probability mass never tank. So, the probability mass never tank. So, the probability mass never
change.… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 14:
00 | in math in this way. So this is like in math in this way. So this is like
basically like how much probability gets basically like how much probability gets basically like how much probability gets
cha… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 15:
12 | two assumptions.
>> Uh two assumptions. One is that >> Uh two assumptions. One is that >> Uh two assumptions. One is that
add up to one. The second one is that add up to one. The second one is that ad… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 16:
04 | thing that like flow out
>> as physically inclined. >> as physically inclined. >> as physically inclined.
>> It is okay. >> It is okay. >> It is okay.
>> I'm just a civil engineer. >> I'm just a civil… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 17:
08 | the divergence connection again the divergence connection again
>> yes I I think I missed a P here actually >> yes I I think I missed a P here actually >> yes I I think I missed a P here actually
by t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 18:
21 | that like so this is the correct that like so this is the correct
divergence. Sorry that that that one I divergence. Sorry that that that one I divergence. Sorry that that that one I
missed P. Um but … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 19:
13 | the log probability of of the log
density. Okay. And this OD is actually density. Okay. And this OD is actually density. Okay. And this OD is actually
equal to so the derivative of the total equal to … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 20:
52 | So exactly so so
how do you train your continuous how do you train your continuous how do you train your continuous
normalizing flow from here? What do we normalizing flow from here? What do we normal… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 22:
04 | we actually need to do numerical
integration right. But this is at integration right. But this is at integration right. But this is at
training time. So that just means that training time. So that jus… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 23:
07 | numerical integration, right? So it
still has a lot of steps. Can we do still has a lot of steps. Can we do still has a lot of steps. Can we do
something that doesn't require a lot of something that d… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 24:
11 | just need to make sure that the velocity
actually match with the ground truth actually match with the ground truth actually match with the ground truth
velocity. velocity. velocity.
So you just do L2 … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 25:
12 | changing every day be like actually I
think that the the storm is moving north think that the the storm is moving north think that the the storm is moving north
or maybe not. We don't know, right? So … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 25:
58 | Yeah. That's right. That's right. You
just go straight. just go straight. just go straight.
Yeah, exactly. Uh so basically this is Yeah, exactly. Uh so basically this is Yeah, exactly. Uh so basically… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 26:
57 | just start from some prior which we can just start from some prior which we can
choose. So it doesn't matter we can choose. So it doesn't matter we can choose. So it doesn't matter we can
choose the p… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 27:
43 | the the data point then you can get a
marginal marginal marginal
uh yeah and also uh just the boundary uh yeah and also uh just the boundary uh yeah and also uh just the boundary
condition essentially… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 29:
03 | probability path and also just like how probability path and also just like how
like and reweed by how likely in general like and reweed by how likely in general like and reweed by how likely in gener… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 30:
38 | this average is like averaging by like this average is like averaging by like
how likely is the data point given your how likely is the data point given your how likely is the data point given your
cu… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 31:
44 | regularization condition. This can can
can be done. Uh and then uh basically can be done. Uh and then uh basically can be done. Uh and then uh basically
you uh input your um because we know you uh inp… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 32:
46 | can construct can generate the marginal
property path which is what we want. property path which is what we want. property path which is what we want.
Oh, I saw so many confusion. Oh, I saw so many co… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 33:
50 | Think about that. Isn't that pretty Think about that. Isn't that pretty
cool? Okay. Why? Why? Why is that? Okay. cool? Okay. Why? Why? Why is that? Okay. cool? Okay. Why? Why? Why is that? Okay.
So, t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 34:
46 | Like basically you if you swap uh the Like basically you if you swap uh the
dx1 term inside so you swap the order of dx1 term inside so you swap the order of dx1 term inside so you swap the order of
t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 35:
50 | we're compressing the bullet.
>> It's literally the same. It's literally >> It's literally the same. It's literally >> It's literally the same. It's literally
the same. It's equivalent. It's not even … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 36:
53 | Well we first have the the the endpoint
distribution and because we're distribution and because we're distribution and because we're
transforming it straight to the point transforming it straight to t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 37:
52 | >> Yeah. No no this is what we chose. This
is a design choice. So that so this is a design choice. So that so this is a design choice. So that so this
thing is generalizable to any kind of thing is ge… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 39:
09 | we can directly sample from
>> therefor you >> like previous lectures like gausian >> like previous lectures like gausian
assumption is used like this closure on assumption is used like this closure o… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 40:
22 | we talked about stuff when we u when we
actually uh decrease the the velocity at actually uh decrease the the velocity at actually uh decrease the the velocity at
which we adding noise. which we addin… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 41:
26 | this particular path. Only on this this particular path. Only on this
particular x0 to x1 path. But it's not particular x0 to x1 path. But it's not particular x0 to x1 path. But it's not
in general. i… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 42:
24 | >> because if you look at the if you look >> because if you look at the if you look
at the this the the left hand side right at the this the the left hand side right at the this the the left hand side… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 43:
16 | literally the same conference and they
all got into the same conference. all got into the same conference. all got into the same conference.
So yeah, so the first group is a group So yeah, so the firs… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 44:
00 | but you may have this question, right? I
feel like some people are having this feel like some people are having this feel like some people are having this
question but they didn't ask. So like questio… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 44:
58 | do is now let's say you you after
training with the linear interpolation training with the linear interpolation training with the linear interpolation
you already got this coupling. So the you already… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 45:
50 | right? So you can map from one point in right? So you can map from one point in
P 0 to another point in P1 and then you P 0 to another point in P1 and then you P 0 to another point in P1 and then you
… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 46:
34 | because you don't know like basically if
you if you only do one step there can be you if you only do one step there can be you if you only do one step there can be
like a lot of um uh uh like like lik… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 47:
37 | >> The third image is the linear >> The third image is the linear
interpolation of the new coupling. interpolation of the new coupling. interpolation of the new coupling.
>> That what is that doing? >… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 48:
17 | connected this side to that side and connected this side to that side and
that side to this side. There's no that side to this side. There's no that side to this side. There's no
there's no this anymo… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 48:
56 | perfectly trained into this process,
>> then you do it again. Yeah. Yeah. So, >> then you do it again. Yeah. Yeah. So, >> then you do it again. Yeah. Yeah. So,
this is just you you get it this is just… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 49:
44 | well. you're just like kind of random
random walking a little bit. Whereas for random walking a little bit. Whereas for random walking a little bit. Whereas for
flow matching, you don't really do flow… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 50:
27 | question is is flow matching also the question is is flow matching also the
same thing because if you think about it same thing because if you think about it same thing because if you think about it
y… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 51:
12 | can write everything as if it is flow can write everything as if it is flow
matching as well and then you can matching as well and then you can matching as well and then you can
actually just recover … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 52:
03 | still need to take care of that part.
But it turns out that we actually do But it turns out that we actually do But it turns out that we actually do
know how to take care of that part. And know how to… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 52:
44 | OD, right? So this is the just um say we OD, right? So this is the just um say we
have this particular OD and it turns out have this particular OD and it turns out have this particular OD and it turns… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 53:
30 | change for the probability means that
the marginalss are all the same. So this the marginalss are all the same. So this the marginalss are all the same. So this
just means that now if you have your ju… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 54:
36 | >> That's a great question. Um so for >> That's a great question. Um so for
example, I don't know if you guys example, I don't know if you guys example, I don't know if you guys
familiar with the RL. … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 55:
40 | Use the use the corresponding PF OD of Use the use the corresponding PF OD of
the SDE. Right? So you just plug it in the SDE. Right? So you just plug it in the SDE. Right? So you just plug it in
and t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 57:
07 | >> A lot of uh uh lot of PD getting
converted to OD. Is it are we just doing converted to OD. Is it are we just doing converted to OD. Is it are we just doing
it because it's simpler or do we you it b… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 57:
53 | diversity but but yes so like
essentially um if you're sampling from a essentially um if you're sampling from a essentially um if you're sampling from a
SDE then it's going to be yeah because SDE then… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 59:
06 | sampling chain through your by by by sampling chain through your by by by
backward integrating your velocity and backward integrating your velocity and backward integrating your velocity and
then duri… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 60:
12 | you know just like you know you just add
something divide something you it's the something divide something you it's the something divide something you it's the
same thing. Okay. same thing. Okay. sam… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 61:
01 | next week we're gonna be able to see
like like like
uh different options to further uh different options to further uh different options to further
improve what we have learned. Um so for improve what… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 61:
41 | the generation faster? And we're going
to learn uh to do it with or without to learn uh to do it with or without to learn uh to do it with or without
training retraining of the model. And training ret… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 00:
04 | Hello everyone and welcome to the next Hello everyone and welcome to the next
lecture of the course principles of lecture of the course principles of lecture of the course principles of
diffusion mode… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 01:
27 | Similarly, every single point in the box
is following a specific trajectory is following a specific trajectory is following a specific trajectory
and the particles are moving as and the particles are … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 02:
49 | with not just space but it is changing with not just space but it is changing
with time also. [snorts] Why it is with time also. [snorts] Why it is with time also. [snorts] Why it is
changing with spa… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 03:
44 | Now we approach another person. Let's Now we approach another person. Let's
call that person uh detective number call that person uh detective number call that person uh detective number
two. And we a… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 04:
36 | from trajectories and then calculating from trajectories and then calculating
the velocity field. [snorts] Now they the velocity field. [snorts] Now they the velocity field. [snorts] Now they
are thin… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 06:
04 | which is a specific point in space. which is a specific point in space.
[snorts] Let's say this is x0. Now what is x1? X1 is equal to Now what is x1? X1 is equal to
X0 + delta X X0 + delta X X0 + delt… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 07:
11 | particles. We have looked at their particles. We have looked at their
trajectories. We have looked at the trajectories. We have looked at the trajectories. We have looked at the
mysterious force that … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 08:
08 | their trajectories
and you collect all these trajectories and you collect all these trajectories and you collect all these trajectories
and you call them as a flow map or just and you call them as a f… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 09:
01 | can get a fair idea about what the flow can get a fair idea about what the flow
is representing is representing is representing
by by by
having this square grid and having this square grid and having … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 09:
50 | flow models in the first place. So far
whatever we have discussed can be whatever we have discussed can be whatever we have discussed can be
understood by a physics student easily understood by a phys… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 10:
47 | okay generate a new image of a cat.
Now the way you solve this problem is Now the way you solve this problem is Now the way you solve this problem is
you try to find a distribution you try to find a d… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 11:
50 | a gshian distribution that is noise and
you end up with the actual distribution. you end up with the actual distribution. you end up with the actual distribution.
So the path that we are going to foll… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 12:
55 | in a way which is similar to how the in a way which is similar to how the
particles are moving in space and time. particles are moving in space and time. particles are moving in space and time.
What d… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 14:
00 | So we are casting the deep generative
modeling problem modeling problem modeling problem
as a problem where we are trying to as a problem where we are trying to as a problem where we are trying to
pre… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 15:
05 | [snorts] and it depends on time as well.
So this is exactly the same equation So this is exactly the same equation So this is exactly the same equation
that we looked at before. that we looked at befo… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 15:
54 | to predict this. So this is approximated
by a neural network. by a neural network. by a neural network.
Okay. And these are the conditions we Okay. And these are the conditions we Okay. And these are … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 16:
56 | distribution that we want to predict in
the first place. the first place. the first place.
And it turns out that this is at the end And it turns out that this is at the end And it turns out that this … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 17:
52 | But what if you start with a gshian But what if you start with a gshian
distribution and you end up with distribution and you end up with distribution and you end up with
something that does not repre… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 18:
45 | is the price of the house. In this case, is the price of the house. In this case,
we want to minimize the loss function so we want to minimize the loss function so we want to minimize the loss functio… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 19:
42 | this target is not very easy. In fact we
have absolutely no idea what this target have absolutely no idea what this target have absolutely no idea what this target
is. is. is.
And that is going to be … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 20:
46 | uh how the training target is predicted uh how the training target is predicted
in the first place. in the first place. in the first place.
And whatever I'm going to discuss next And whatever I'm goin… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 21:
51 | breed.
So we have different So we have different So we have different
we have a probability distribution that we have a probability distribution that we have a probability distribution that
you know t… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 22:
46 | going to follow next and uh we are going
to go through a little bit of to go through a little bit of to go through a little bit of
mathematics right now so please don't be mathematics right now so ple… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 23:
38 | Okay. So, first concept that we will Okay. So, first concept that we will
discuss is called as So let's let's try to understand what is So let's let's try to understand what is
the meaning of [snorts]… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 24:
51 | to something I do know even if it is
like just one simple single data point like just one simple single data point like just one simple single data point
[snorts] and that is exactly what [snorts] and… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 25:
51 | know the data which is given to us. We
know this initial distribution also. know this initial distribution also. know this initial distribution also.
So it is very rational to think that we So it is v… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 27:
05 | go to the point Z. So it is conditioned
on the point Z. on the point Z.
So what can I do so that each individual So what can I do so that each individual So what can I do so that each individual
distr… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 28:
46 | such that at every time step I will have such that at every time step I will have
a gshian and a gshian and a gshian and
the probability distribution will change the probability distribution will chan… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 29:
39 | probability path which is linear in probability path which is linear in
time. time. time.
So the mean increases from zero to the So the mean increases from zero to the So the mean increases from zero … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 30:
39 | going to generate those trajectories.
uh and this is exactly what we are going uh and this is exactly what we are going uh and this is exactly what we are going
to do next. But before we move, I want … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 31:
23 | anyways uh for now just think about this
as being a conditional probability path as being a conditional probability path as being a conditional probability path
which is conditioned on the point Z whi… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 32:
15 | single point that is the probability
path and we want to know at every single path and we want to know at every single path and we want to know at every single
time step in this transition how does ti… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 33:
17 | trajectories the reverse problem trajectories the reverse problem
we simply sample the initial point which we simply sample the initial point which we simply sample the initial point which
was x0 in o… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 34:
06 | this transformation? There is a vector
field which is responsible for this field which is responsible for this field which is responsible for this
transformation transformation
and [snorts] and [snort… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 35:
06 | every single point the velocity vector
is pointed towards them. And And
for points which are far away the for points which are far away the for points which are far away the
vectors are longer in magn… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 36:
05 | this is because as as as time proceeds this is because as as as time proceeds
we probably want to uh move the velocity we probably want to uh move the velocity we probably want to uh move the velocity… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 36:
56 | we don't need anything else using this
formula we can simply calculate the formula we can simply calculate the formula we can simply calculate the
trajectories of each and every point so trajectories … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 37:
46 | Now this u of t is is called as Now this u of t is is called as
conditional So you you might look at this and think So you you might look at this and think
uh well if you look at the training uh well … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 38:
40 | our target and train our neural network
to predict it for every single point to predict it for every single point to predict it for every single point
zed. zed. zed.
It turns out that what we will do … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 39:
37 | velocity vector field which is pointed
from our data to the data point where we from our data to the data point where we from our data to the data point where we
want to go. So it's it's a it's arrow … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 40:
32 | final flow that you get the the final
trajectories that you get those are the trajectories that you get those are the trajectories that you get those are the
marginal probability paths. marginal proba… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 41:
24 | to us. So here if you see to us. So here if you see
the final distribution that we get which the final distribution that we get which the final distribution that we get which
is which are these um is … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 42:
50 | want to reach which is a sample of eight
washians washians washians
and initially we start with a random and initially we start with a random and initially we start with a random
noise. noise. noise.
… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 43:
45 | Now uh here let's play this
piece of code to understand this piece of code to understand this piece of code to understand this
u and you'll probably get something like and you'll probably get somethin… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 44:
39 | in our target. But uh the the issue is
that we do not have access to this that we do not have access to this that we do not have access to this
because we only have access to our because we only have … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 45:
39 | by z minus x divided by 1 minus t. by z minus x divided by 1 minus t.
[snorts] So these these these field [snorts] So these these these field [snorts] So these these these field
lines are almost linea… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 46:
30 | we do not have access to infinite
samples. we have access to finite samples. we have access to finite samples. we have access to finite
samples samples samples
and uh this this uh credit goes to uh an… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 47:
18 | understand what we are doing here.
Essentially we are treating data Essentially we are treating data Essentially we are treating data
generation as a flow problem. We know generation as a flow problem… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 48:
17 | field is impossibly complex.
You cannot write down an equation that You cannot write down an equation that You cannot write down an equation that
defines what a Giblly movie looks like. defines what a… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 49:
06 | be true and it does actually work. We we be true and it does actually work. We we
have seen this in other uh aspects of have seen this in other uh aspects of have seen this in other uh aspects of
deep… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 49:
54 | uh I I hope this lecture was interesting
for all of you. I haven't yet given the for all of you. I haven't yet given the for all of you. I haven't yet given the
final reveal. Uh I'm I'm delaying it fi… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 50:
45 | this this came out last year this this came out last year
and this is based on uh flow matching. and this is based on uh flow matching. and this is based on uh flow matching.
So here essentially what … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 51:
32 | is which is the action expert here which
is a it's based on flow matching and uh is a it's based on flow matching and uh is a it's based on flow matching and uh
the training target that you see here i… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 52:
09 | lecture which is going to be like a
magician's reveal. Thank you. |