cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 00:
03 | Okay, so let's get started. uh so far uh Okay, so let's get started. uh so far uh
our last class we have talked about you our last class we have talked about you our last class we have talked about yo… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 00:
48 | Okay. Uh so this is just like a quick
like reminder of like like reminder of like like reminder of like
what happened next last class. So what happened next last class. So what happened next last clas… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 01:
39 | lastly we talked about like the easiest lastly we talked about like the easiest
way to do discrete diffusion which is way to do discrete diffusion which is way to do discrete diffusion which is
mass d… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 02:
46 | probability path, right? to go from like
some noise or like some the full mass to some noise or like some the full mass to some noise or like some the full mass to
clean data, right? Yeah, basically t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 03:
32 | let let's look at them in a in the
discrete settings. So last time we kind discrete settings. So last time we kind discrete settings. So last time we kind
of skipped through this part uh or of skipped… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 04:
23 | distribution, right? So the probability
the sum of the probability over all the sum of the probability over all the sum of the probability over all
possible states or all possible tokens possible stat… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 05:
13 | transitions should add up to zero. And
this is like and this is literally just this is like and this is literally just this is like and this is literally just
a con conversation uh conservation of a c… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 06:
09 | likely that you're going to go basically
it's like how how how likely that you're it's like how how how likely that you're it's like how how how likely that you're
going to go jump from this state to … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 07:
03 | It's a velocity, right? Essentially. So It's a velocity, right? Essentially. So
it's basically just saying that like um it's basically just saying that like um it's basically just saying that like um
… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 07:
53 | this is also why we're trying to do
reparameterization trick yeah but that's reparameterization trick yeah but that's reparameterization trick yeah but that's
a great question. All right. Anything a g… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 08:
58 | this state that's it right this is like
super uh very intuitive I hope is super uh very intuitive I hope is super uh very intuitive I hope is
All good. What can we observe here? Can we write What can … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 10:
17 | not equal anymore. So just sum of all
the z all the possible z's and then qzx the z all the possible z's and then qzx the z all the possible z's and then qzx
ptz right and z could be equal to x as ptz… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 11:
21 | oiler, right? So you just get you you
just literally plug everything in here just literally plug everything in here just literally plug everything in here
and then you'll get this thing and the and th… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 12:
15 | like the the core of the the story in
all the discrete diffusion papers is all the discrete diffusion papers is all the discrete diffusion papers is
that like no matter what you do try to that like no… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 13:
03 | just use cross entropy to learn this,
right? Because we have the ground truth right? Because we have the ground truth right? Because we have the ground truth
clean data. So we just need to use cross c… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 14:
01 | flips right. So you start from t t flips right. So you start from t t
equals z which and then you sample from equals z which and then you sample from equals z which and then you sample from
your prior… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 14:
45 | you just increment the time and then you
do it over over again until you reach do it over over again until you reach do it over over again until you reach
the end point. Okay, make sense? This this is… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 15:
42 | things that we developed somehow but
basically there are two discreful basically there are two discreful basically there are two discreful
matching papers uh this time they are matching papers uh this… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 16:
22 | know but anyway it's like really like I
really hope this guy get rich and famous really hope this guy get rich and famous really hope this guy get rich and famous
soon. Uh anyway and then the second o… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 17:
05 | of learn the rate matrix but we don't of learn the rate matrix but we don't
actually learn the rate matrix directly actually learn the rate matrix directly actually learn the rate matrix directly
we w… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 17:
50 | your par paragraph is in order to build
this model and this is just like super this model and this is just like super this model and this is just like super
like not good right this is a very very lik… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 18:
36 | like it's the same problem as the fixed
length thing, right? Yeah. length thing, right? Yeah. length thing, right? Yeah.
>> I heard this from like who's doing >> I heard this from like who's doing >> … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 19:
20 | right. So let's take a look at it. Uh right. So let's take a look at it. Uh
yeah. So essentially variable length you yeah. So essentially variable length you yeah. So essentially variable length you
s… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 19:
57 | like something that we can think about,
right? And it's like now we have like right? And it's like now we have like right? And it's like now we have like
two um like frameworks that like sort of two u… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 20:
56 | >> Yeah. What else? >> Yeah. What else?
>> Inserting at but I saw that you can do >> Inserting at but I saw that you can do >> Inserting at but I saw that you can do
decoding in blocks like you can pr… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 21:
29 | >> Okay, so tatting at the end. So you can >> Okay, so tatting at the end. So you can
discard some of the tokens. Yeah, that discard some of the tokens. Yeah, that discard some of the tokens. Yeah, th… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 22:
15 | paragraph and for the next diffusion
process that paragraph is that one process that paragraph is that one process that paragraph is that one
element for you. You noise the paragraph element for you. … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 23:
09 | the same thing and that this is like
this is corresponding to one paper that this is corresponding to one paper that this is corresponding to one paper that
we're going to talk about but not now we're… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 23:
49 | different tokens in between tokens,
right? I don't need to insert at the right? I don't need to insert at the right? I don't need to insert at the
end. Why do I have to insert at the end? end. Why do … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 24:
31 | >> Does this work in the diffusion case?
>> Good question. I I'm gonna talk about >> Good question. I I'm gonna talk about >> Good question. I I'm gonna talk about
it. Okay. So, how do we even model t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 25:
15 | you use it to to to essentially
construct construct the the ground truth construct construct the the ground truth construct construct the the ground truth
of like what this like what the full of like … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 25:
57 | right? What is this other thing?
>> The position or the whether to generate >> The position or the whether to generate >> The position or the whether to generate
it between these it between these it b… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 26:
52 | basically if we're missing tokens and basically if we're missing tokens and
like essentially like essentially like essentially
note that like it's more likely or like note that like it's more likely o… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 27:
45 | what you're predicting and then the
second objective from your model is second objective from your model is second objective from your model is
predict what tokens are missing so this predict what tok… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 28:
26 | like I don't know.
>> Yeah. >> Yeah.
>> Yeah. But basically what Okay. So >> Yeah. But basically what Okay. So >> Yeah. But basically what Okay. So
basically what happens in the model or basically wha… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 29:
10 | >> Yeah. Yeah. So basically what's
happening in what's hap what's happening happening in what's hap what's happening happening in what's hap what's happening
in like implementation is that you are in … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 29:
55 | is that is missing out of all the tokens is that is missing out of all the tokens
that are missing? Right. Okay. So these that are missing? Right. Okay. So these that are missing? Right. Okay. So thes… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 30:
48 | missing versus you know how many tokens
you predict to be missing and then the you predict to be missing and then the you predict to be missing and then the
second thing is uh like literally just seco… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 31:
48 | flow
And why is it a flow model? Well, it's And why is it a flow model? Well, it's And why is it a flow model? Well, it's
actually because this thing like the actually because this thing like the actu… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 32:
35 | another thing then you should be doing another thing then you should be doing
this operation where it's a ratio or this operation where it's a ratio or this operation where it's a ratio or
like a rate… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 33:
23 | divergence that is basically just a
generalized version of KL. Uh but this generalized version of KL. Uh but this generalized version of KL. Uh but this
is like too complicated. We're not going is lik… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 34:
11 | actually like already good enough for
whatever you want to do. So you can whatever you want to do. So you can whatever you want to do. So you can
literally just train a insertion based literally just … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 34:
56 | we kind of fixed a veryable
length uh problem with insertion based length uh problem with insertion based length uh problem with insertion based
generation. Uh we have other things to generation. Uh w… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 35:
40 | So like basically this thing is very
nice because you can um like essentially nice because you can um like essentially nice because you can um like essentially
you can do paralyze generation like you … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 36:
20 | that you have generated like in your
previous blocks. The only thing you need previous blocks. The only thing you need previous blocks. The only thing you need
to like do birectional attention to is t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 37:
00 | people think about this. Um but yeah, so people think about this. Um but yeah, so
this is like a nice way to do you know this is like a nice way to do you know this is like a nice way to do you know
t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 37:
50 | >> not in the current block but but we can
K catch all the previous blocks. >> No because we don't we haven't generated >> No because we don't we haven't generated
that yet, right? So like the block a… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 38:
29 | of like also like but but but but if you
have too small of a blog then it just have too small of a blog then it just have too small of a blog then it just
Yeah. It doesn't really it just is just Yeah.… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 39:
16 | in I think. Yeah. in I think. Yeah.
>> Do you think the second one is >> Do you think the second one is >> Do you think the second one is
um like it's it's the it's a problem of um like it's it's the … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 40:
11 | That's correct. Yeah.
>> For the for the second case, the mass >> For the for the second case, the mass >> For the for the second case, the mass
diffusion, why can't we do something diffusion, why can… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 40:
57 | Anything else? Yep.
>> Can you overhand why mass diffusion is >> Can you overhand why mass diffusion is >> Can you overhand why mass diffusion is
lower? lower? lower?
>> Uh okay. Who wants to tell who… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 41:
49 | but it doesn't mean that you you cannot
train a good enough diffusion model to train a good enough diffusion model to train a good enough diffusion model to
blm that's not the case it's just that blm … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 42:
45 | the the optimal that you're going to get the the optimal that you're going to get
from whatever algorithm that they from whatever algorithm that they from whatever algorithm that they
develop will loo… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 43:
32 | this is not image and text but like
we're going to look at image and text we're going to look at image and text we're going to look at image and text
later. But for those of you who are in later. But … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 44:
31 | >> What are we cogenerating?
>> So, we're co-generating something uh >> So, we're co-generating something uh >> So, we're co-generating something uh
discreet and something uh continuous at discreet an… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 45:
15 | like alternating between the two that's like alternating between the two that's
pretty much that right and like pretty much that right and like pretty much that right and like
basically And then if yo… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 46:
03 | well right so what you can do is you can well right so what you can do is you can
uh have like just like I think some uh have like just like I think some uh have like just like I think some
someone al… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 46:
50 | >> I mean the problem is like this is a >> I mean the problem is like this is a
research project right so it's like very research project right so it's like very research project right so it's like ve… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 47:
37 | and then the the the image for example
like flower or like cat and then and like flower or like cat and then and like flower or like cat and then and
then the the the the image are going to then the t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 48:
13 | text and like image at the same time text and like image at the same time
because like you start from nothing and because like you start from nothing and because like you start from nothing and
then y… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 48:
49 | the way to become omni but this is like
just like two two modalities I guess. just like two two modalities I guess. just like two two modalities I guess.
Yeah. Yeah. Omni model just means that Yeah. Y… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 49:
50 | you cannot access the ponapto anymore,
it's still going to be on YouTube. Uh I it's still going to be on YouTube. Uh I it's still going to be on YouTube. Uh I
kind of do some editing uh for the kind o… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 50:
25 | your turn to take what you've learned
into the real world so I will highly into the real world so I will highly into the real world so I will highly
recommend if you are proud of what recommend if you… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 51:
05 | something like that. I think it could be something like that. I think it could be
a good experience if you guys are a good experience if you guys are a good experience if you guys are
interested in ge… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 51:
39 | research slash you know when you go to
the AGI company you know make the AGI a the AGI company you know make the AGI a the AGI company you know make the AGI a
diffusion model please we don't want to d… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 52:
25 | >> padding tokens and tokens after the OS,
right? right?
>> Yeah. >> Yeah.
>> Uh so one thing I implemented for >> Uh so one thing I implemented for >> Uh so one thing I implemented for
homework 3 and… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 53:
06 | I thought that's what they did in I thought that's what they did in
refflow but maybe not. If it's not then refflow but maybe not. If it's not then refflow but maybe not. If it's not then
this is a go… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 53:
42 | than a 2 RF model because the only than a 2 RF model because the only
errors accumulate. errors accumulate. errors accumulate.
>> Yeah. No, that's your homework. Four. >> Yeah. No, that's your homewor… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 54:
19 | in general I think that's a better idea
than the normal the the the way that than the normal the the the way that than the normal the the the way that
they they did. Okay. Any other question? they the… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 55:
03 | >> Yeah. Yeah. So actually there multiple >> Yeah. Yeah. So actually there multiple
uh so the most famous one in the uh so the most famous one in the uh so the most famous one in the
robotics world is… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 55:
49 | gradient you should apply from those
buffer data to your current time to one buffer data to your current time to one buffer data to your current time to one
of the time that you sampled. Um so this of… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 56:
25 | in that framework but you can turn you in that framework but you can turn you
can essentially turn an S OD like a flow can essentially turn an S OD like a flow can essentially turn an S OD like a flow… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 57:
06 | basically is there any other ways that
we can do it besides turning into a SDE we can do it besides turning into a SDE we can do it besides turning into a SDE
um yeah so like let's hope we can get um … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 57:
58 | essentially you just sort of like uh
they sort of extend the flow map uh uh they sort of extend the flow map uh uh they sort of extend the flow map uh uh
what you call framework into the what you call… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 58:
37 | Uh, so let's see if he actually post it
online this week. Uh, but yeah, I don't online this week. Uh, but yeah, I don't online this week. Uh, but yeah, I don't
actually know how how he did it, but act… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 59:
19 | of similar to uh one of the other
previous uh framework and equilibrium previous uh framework and equilibrium previous uh framework and equilibrium
model. So equilibrium model is something model. So e… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 60:
02 | or the underlying uh, you know, uh, math
is a little bit related, but like this is a little bit related, but like this is a little bit related, but like this
is definitely something new. I think is de… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 00:
02 | So last time we learned about how So last time we learned about how
diffusion turn noise into data by diffusion turn noise into data by diffusion turn noise into data by
gradually adding noise and gra… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 00:
52 | by an SDE with this score function and
then we can just do score matching to then we can just do score matching to then we can just do score matching to
learn uh how to solve the uh the learn uh how t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 01:
41 | That's exactly correct. Okay. Why don't
we just do linear interpolation of the we just do linear interpolation of the we just do linear interpolation of the
two things, right? Um so exact how two thin… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 02:
26 | this trajectory in this way. Um yeah so
remember the same noise same data. remember the same noise same data. remember the same noise same data.
Then learning the transformation along Then learning th… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 03:
28 | progress. So like 0.25 time um times uh progress. So like 0.25 time um times uh
data minus noise, right? Uh so basically data minus noise, right? Uh so basically data minus noise, right? Uh so basical… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 04:
20 | this is literally what we need to learn, this is literally what we need to learn,
right? This is like literally the only right? This is like literally the only right? This is like literally the only
t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 05:
17 | Right? How do you progress through the
chain? You just compute the displacement chain? You just compute the displacement chain? You just compute the displacement
for x and then directly apply the for … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 06:
26 | full matching. Class is over. Hey,
no, not really. Okay, fine. Uh but no, not really. Okay, fine. Uh but no, not really. Okay, fine. Uh but
basically the the question that we're basically the the ques… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 07:
10 | people start to de develop this thing.
Okay. So basically back in 2018 there Okay. So basically back in 2018 there Okay. So basically back in 2018 there
this like pretty seminal paper um uh this like … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 08:
05 | can just solve this OD by integrating can just solve this OD by integrating
through uh the velocity and then you through uh the velocity and then you through uh the velocity and then you
you'll be abl… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 08:
55 | Anyone else I haven't talked to today?
Nobody remember what is normalizing Nobody remember what is normalizing Nobody remember what is normalizing
flow? >> You can reconstruct the process. >> You can … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 09:
50 | invertible transformation because invertible transformation because
everything is deterministic. everything is deterministic. everything is deterministic.
Okay, any question? Okay, any question? Okay,… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 11:
01 | no new mass uh is is going to get added
and no mass is going to disappear and no mass is going to disappear and no mass is going to disappear
getting dropped. So like translate it getting dropped. So … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 11:
50 | probability flux. So imagine a flux is
like the amount of flow the amount of like the amount of flow the amount of like the amount of flow the amount of
water uh per unit per time that's going water u… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 12:
55 | in a container. It's in like a contained in a container. It's in like a contained
tank. So, the probability mass never tank. So, the probability mass never tank. So, the probability mass never
change.… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 14:
00 | in math in this way. So this is like in math in this way. So this is like
basically like how much probability gets basically like how much probability gets basically like how much probability gets
cha… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 15:
12 | two assumptions.
>> Uh two assumptions. One is that >> Uh two assumptions. One is that >> Uh two assumptions. One is that
add up to one. The second one is that add up to one. The second one is that ad… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 16:
04 | thing that like flow out
>> as physically inclined. >> as physically inclined. >> as physically inclined.
>> It is okay. >> It is okay. >> It is okay.
>> I'm just a civil engineer. >> I'm just a civil… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 17:
08 | the divergence connection again the divergence connection again
>> yes I I think I missed a P here actually >> yes I I think I missed a P here actually >> yes I I think I missed a P here actually
by t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 18:
21 | that like so this is the correct that like so this is the correct
divergence. Sorry that that that one I divergence. Sorry that that that one I divergence. Sorry that that that one I
missed P. Um but … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 19:
13 | the log probability of of the log
density. Okay. And this OD is actually density. Okay. And this OD is actually density. Okay. And this OD is actually
equal to so the derivative of the total equal to … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 20:
52 | So exactly so so
how do you train your continuous how do you train your continuous how do you train your continuous
normalizing flow from here? What do we normalizing flow from here? What do we normal… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 22:
04 | we actually need to do numerical
integration right. But this is at integration right. But this is at integration right. But this is at
training time. So that just means that training time. So that jus… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 23:
07 | numerical integration, right? So it
still has a lot of steps. Can we do still has a lot of steps. Can we do still has a lot of steps. Can we do
something that doesn't require a lot of something that d… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 24:
11 | just need to make sure that the velocity
actually match with the ground truth actually match with the ground truth actually match with the ground truth
velocity. velocity. velocity.
So you just do L2 … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 25:
12 | changing every day be like actually I
think that the the storm is moving north think that the the storm is moving north think that the the storm is moving north
or maybe not. We don't know, right? So … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 25:
58 | Yeah. That's right. That's right. You
just go straight. just go straight. just go straight.
Yeah, exactly. Uh so basically this is Yeah, exactly. Uh so basically this is Yeah, exactly. Uh so basically… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 26:
57 | just start from some prior which we can just start from some prior which we can
choose. So it doesn't matter we can choose. So it doesn't matter we can choose. So it doesn't matter we can
choose the p… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 27:
43 | the the data point then you can get a
marginal marginal marginal
uh yeah and also uh just the boundary uh yeah and also uh just the boundary uh yeah and also uh just the boundary
condition essentially… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 29:
03 | probability path and also just like how probability path and also just like how
like and reweed by how likely in general like and reweed by how likely in general like and reweed by how likely in gener… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 30:
38 | this average is like averaging by like this average is like averaging by like
how likely is the data point given your how likely is the data point given your how likely is the data point given your
cu… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 31:
44 | regularization condition. This can can
can be done. Uh and then uh basically can be done. Uh and then uh basically can be done. Uh and then uh basically
you uh input your um because we know you uh inp… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 32:
46 | can construct can generate the marginal
property path which is what we want. property path which is what we want. property path which is what we want.
Oh, I saw so many confusion. Oh, I saw so many co… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 33:
50 | Think about that. Isn't that pretty Think about that. Isn't that pretty
cool? Okay. Why? Why? Why is that? Okay. cool? Okay. Why? Why? Why is that? Okay. cool? Okay. Why? Why? Why is that? Okay.
So, t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 34:
46 | Like basically you if you swap uh the Like basically you if you swap uh the
dx1 term inside so you swap the order of dx1 term inside so you swap the order of dx1 term inside so you swap the order of
t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 35:
50 | we're compressing the bullet.
>> It's literally the same. It's literally >> It's literally the same. It's literally >> It's literally the same. It's literally
the same. It's equivalent. It's not even … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 36:
53 | Well we first have the the the endpoint
distribution and because we're distribution and because we're distribution and because we're
transforming it straight to the point transforming it straight to t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 37:
52 | >> Yeah. No no this is what we chose. This
is a design choice. So that so this is a design choice. So that so this is a design choice. So that so this
thing is generalizable to any kind of thing is ge… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 39:
09 | we can directly sample from
>> therefor you >> like previous lectures like gausian >> like previous lectures like gausian
assumption is used like this closure on assumption is used like this closure o… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 40:
22 | we talked about stuff when we u when we
actually uh decrease the the velocity at actually uh decrease the the velocity at actually uh decrease the the velocity at
which we adding noise. which we addin… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 41:
26 | this particular path. Only on this this particular path. Only on this
particular x0 to x1 path. But it's not particular x0 to x1 path. But it's not particular x0 to x1 path. But it's not
in general. i… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 42:
24 | >> because if you look at the if you look >> because if you look at the if you look
at the this the the left hand side right at the this the the left hand side right at the this the the left hand side… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 43:
16 | literally the same conference and they
all got into the same conference. all got into the same conference. all got into the same conference.
So yeah, so the first group is a group So yeah, so the firs… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 44:
00 | but you may have this question, right? I
feel like some people are having this feel like some people are having this feel like some people are having this
question but they didn't ask. So like questio… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 44:
58 | do is now let's say you you after
training with the linear interpolation training with the linear interpolation training with the linear interpolation
you already got this coupling. So the you already… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 45:
50 | right? So you can map from one point in right? So you can map from one point in
P 0 to another point in P1 and then you P 0 to another point in P1 and then you P 0 to another point in P1 and then you
… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 46:
34 | because you don't know like basically if
you if you only do one step there can be you if you only do one step there can be you if you only do one step there can be
like a lot of um uh uh like like lik… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 47:
37 | >> The third image is the linear >> The third image is the linear
interpolation of the new coupling. interpolation of the new coupling. interpolation of the new coupling.
>> That what is that doing? >… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 48:
17 | connected this side to that side and connected this side to that side and
that side to this side. There's no that side to this side. There's no that side to this side. There's no
there's no this anymo… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 48:
56 | perfectly trained into this process,
>> then you do it again. Yeah. Yeah. So, >> then you do it again. Yeah. Yeah. So, >> then you do it again. Yeah. Yeah. So,
this is just you you get it this is just… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 49:
44 | well. you're just like kind of random
random walking a little bit. Whereas for random walking a little bit. Whereas for random walking a little bit. Whereas for
flow matching, you don't really do flow… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 50:
27 | question is is flow matching also the question is is flow matching also the
same thing because if you think about it same thing because if you think about it same thing because if you think about it
y… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 51:
12 | can write everything as if it is flow can write everything as if it is flow
matching as well and then you can matching as well and then you can matching as well and then you can
actually just recover … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 52:
03 | still need to take care of that part.
But it turns out that we actually do But it turns out that we actually do But it turns out that we actually do
know how to take care of that part. And know how to… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 52:
44 | OD, right? So this is the just um say we OD, right? So this is the just um say we
have this particular OD and it turns out have this particular OD and it turns out have this particular OD and it turns… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 53:
30 | change for the probability means that
the marginalss are all the same. So this the marginalss are all the same. So this the marginalss are all the same. So this
just means that now if you have your ju… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 54:
36 | >> That's a great question. Um so for >> That's a great question. Um so for
example, I don't know if you guys example, I don't know if you guys example, I don't know if you guys
familiar with the RL. … |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 55:
40 | Use the use the corresponding PF OD of Use the use the corresponding PF OD of
the SDE. Right? So you just plug it in the SDE. Right? So you just plug it in the SDE. Right? So you just plug it in
and t… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 57:
07 | >> A lot of uh uh lot of PD getting
converted to OD. Is it are we just doing converted to OD. Is it are we just doing converted to OD. Is it are we just doing
it because it's simpler or do we you it b… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 57:
53 | diversity but but yes so like
essentially um if you're sampling from a essentially um if you're sampling from a essentially um if you're sampling from a
SDE then it's going to be yeah because SDE then… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 59:
06 | sampling chain through your by by by sampling chain through your by by by
backward integrating your velocity and backward integrating your velocity and backward integrating your velocity and
then duri… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 60:
12 | you know just like you know you just add
something divide something you it's the something divide something you it's the something divide something you it's the
same thing. Okay. same thing. Okay. sam… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 61:
01 | next week we're gonna be able to see
like like like
uh different options to further uh different options to further uh different options to further
improve what we have learned. Um so for improve what… |
cmu-10799-diffusion-flow |
L5: CMU 10799 S26: Lecture 5 - Flow Matching - Diffusion & Flow Matching | 61:
41 | the generation faster? And we're going
to learn uh to do it with or without to learn uh to do it with or without to learn uh to do it with or without
training retraining of the model. And training ret… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 00:
04 | Hello everyone and welcome to the next Hello everyone and welcome to the next
lecture of the course principles of lecture of the course principles of lecture of the course principles of
diffusion mode… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 01:
27 | Similarly, every single point in the box
is following a specific trajectory is following a specific trajectory is following a specific trajectory
and the particles are moving as and the particles are … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 02:
49 | with not just space but it is changing with not just space but it is changing
with time also. [snorts] Why it is with time also. [snorts] Why it is with time also. [snorts] Why it is
changing with spa… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 03:
44 | Now we approach another person. Let's Now we approach another person. Let's
call that person uh detective number call that person uh detective number call that person uh detective number
two. And we a… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 04:
36 | from trajectories and then calculating from trajectories and then calculating
the velocity field. [snorts] Now they the velocity field. [snorts] Now they the velocity field. [snorts] Now they
are thin… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 06:
04 | which is a specific point in space. which is a specific point in space.
[snorts] Let's say this is x0. Now what is x1? X1 is equal to Now what is x1? X1 is equal to
X0 + delta X X0 + delta X X0 + delt… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 07:
11 | particles. We have looked at their particles. We have looked at their
trajectories. We have looked at the trajectories. We have looked at the trajectories. We have looked at the
mysterious force that … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 08:
08 | their trajectories
and you collect all these trajectories and you collect all these trajectories and you collect all these trajectories
and you call them as a flow map or just and you call them as a f… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 09:
01 | can get a fair idea about what the flow can get a fair idea about what the flow
is representing is representing is representing
by by by
having this square grid and having this square grid and having … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 09:
50 | flow models in the first place. So far
whatever we have discussed can be whatever we have discussed can be whatever we have discussed can be
understood by a physics student easily understood by a phys… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 10:
47 | okay generate a new image of a cat.
Now the way you solve this problem is Now the way you solve this problem is Now the way you solve this problem is
you try to find a distribution you try to find a d… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 11:
50 | a gshian distribution that is noise and
you end up with the actual distribution. you end up with the actual distribution. you end up with the actual distribution.
So the path that we are going to foll… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 12:
55 | in a way which is similar to how the in a way which is similar to how the
particles are moving in space and time. particles are moving in space and time. particles are moving in space and time.
What d… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 14:
00 | So we are casting the deep generative
modeling problem modeling problem modeling problem
as a problem where we are trying to as a problem where we are trying to as a problem where we are trying to
pre… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 15:
05 | [snorts] and it depends on time as well.
So this is exactly the same equation So this is exactly the same equation So this is exactly the same equation
that we looked at before. that we looked at befo… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 15:
54 | to predict this. So this is approximated
by a neural network. by a neural network. by a neural network.
Okay. And these are the conditions we Okay. And these are the conditions we Okay. And these are … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 16:
56 | distribution that we want to predict in
the first place. the first place. the first place.
And it turns out that this is at the end And it turns out that this is at the end And it turns out that this … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 17:
52 | But what if you start with a gshian But what if you start with a gshian
distribution and you end up with distribution and you end up with distribution and you end up with
something that does not repre… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 18:
45 | is the price of the house. In this case, is the price of the house. In this case,
we want to minimize the loss function so we want to minimize the loss function so we want to minimize the loss functio… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 19:
42 | this target is not very easy. In fact we
have absolutely no idea what this target have absolutely no idea what this target have absolutely no idea what this target
is. is. is.
And that is going to be … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 20:
46 | uh how the training target is predicted uh how the training target is predicted
in the first place. in the first place. in the first place.
And whatever I'm going to discuss next And whatever I'm goin… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 21:
51 | breed.
So we have different So we have different So we have different
we have a probability distribution that we have a probability distribution that we have a probability distribution that
you know t… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 22:
46 | going to follow next and uh we are going
to go through a little bit of to go through a little bit of to go through a little bit of
mathematics right now so please don't be mathematics right now so ple… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 23:
38 | Okay. So, first concept that we will Okay. So, first concept that we will
discuss is called as So let's let's try to understand what is So let's let's try to understand what is
the meaning of [snorts]… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 24:
51 | to something I do know even if it is
like just one simple single data point like just one simple single data point like just one simple single data point
[snorts] and that is exactly what [snorts] and… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 25:
51 | know the data which is given to us. We
know this initial distribution also. know this initial distribution also. know this initial distribution also.
So it is very rational to think that we So it is v… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 27:
05 | go to the point Z. So it is conditioned
on the point Z. on the point Z.
So what can I do so that each individual So what can I do so that each individual So what can I do so that each individual
distr… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 28:
46 | such that at every time step I will have such that at every time step I will have
a gshian and a gshian and a gshian and
the probability distribution will change the probability distribution will chan… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 29:
39 | probability path which is linear in probability path which is linear in
time. time. time.
So the mean increases from zero to the So the mean increases from zero to the So the mean increases from zero … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 30:
39 | going to generate those trajectories.
uh and this is exactly what we are going uh and this is exactly what we are going uh and this is exactly what we are going
to do next. But before we move, I want … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 31:
23 | anyways uh for now just think about this
as being a conditional probability path as being a conditional probability path as being a conditional probability path
which is conditioned on the point Z whi… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 32:
15 | single point that is the probability
path and we want to know at every single path and we want to know at every single path and we want to know at every single
time step in this transition how does ti… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 33:
17 | trajectories the reverse problem trajectories the reverse problem
we simply sample the initial point which we simply sample the initial point which we simply sample the initial point which
was x0 in o… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 34:
06 | this transformation? There is a vector
field which is responsible for this field which is responsible for this field which is responsible for this
transformation transformation
and [snorts] and [snort… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 35:
06 | every single point the velocity vector
is pointed towards them. And And
for points which are far away the for points which are far away the for points which are far away the
vectors are longer in magn… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 36:
05 | this is because as as as time proceeds this is because as as as time proceeds
we probably want to uh move the velocity we probably want to uh move the velocity we probably want to uh move the velocity… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 36:
56 | we don't need anything else using this
formula we can simply calculate the formula we can simply calculate the formula we can simply calculate the
trajectories of each and every point so trajectories … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 37:
46 | Now this u of t is is called as Now this u of t is is called as
conditional So you you might look at this and think So you you might look at this and think
uh well if you look at the training uh well … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 38:
40 | our target and train our neural network
to predict it for every single point to predict it for every single point to predict it for every single point
zed. zed. zed.
It turns out that what we will do … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 39:
37 | velocity vector field which is pointed
from our data to the data point where we from our data to the data point where we from our data to the data point where we
want to go. So it's it's a it's arrow … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 40:
32 | final flow that you get the the final
trajectories that you get those are the trajectories that you get those are the trajectories that you get those are the
marginal probability paths. marginal proba… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 41:
24 | to us. So here if you see to us. So here if you see
the final distribution that we get which the final distribution that we get which the final distribution that we get which
is which are these um is … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 42:
50 | want to reach which is a sample of eight
washians washians washians
and initially we start with a random and initially we start with a random and initially we start with a random
noise. noise. noise.
… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 43:
45 | Now uh here let's play this
piece of code to understand this piece of code to understand this piece of code to understand this
u and you'll probably get something like and you'll probably get somethin… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 44:
39 | in our target. But uh the the issue is
that we do not have access to this that we do not have access to this that we do not have access to this
because we only have access to our because we only have … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 45:
39 | by z minus x divided by 1 minus t. by z minus x divided by 1 minus t.
[snorts] So these these these field [snorts] So these these these field [snorts] So these these these field
lines are almost linea… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 46:
30 | we do not have access to infinite
samples. we have access to finite samples. we have access to finite samples. we have access to finite
samples samples samples
and uh this this uh credit goes to uh an… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 47:
18 | understand what we are doing here.
Essentially we are treating data Essentially we are treating data Essentially we are treating data
generation as a flow problem. We know generation as a flow problem… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 48:
17 | field is impossibly complex.
You cannot write down an equation that You cannot write down an equation that You cannot write down an equation that
defines what a Giblly movie looks like. defines what a… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 49:
06 | be true and it does actually work. We we be true and it does actually work. We we
have seen this in other uh aspects of have seen this in other uh aspects of have seen this in other uh aspects of
deep… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 49:
54 | uh I I hope this lecture was interesting
for all of you. I haven't yet given the for all of you. I haven't yet given the for all of you. I haven't yet given the
final reveal. Uh I'm I'm delaying it fi… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 50:
45 | this this came out last year this this came out last year
and this is based on uh flow matching. and this is based on uh flow matching. and this is based on uh flow matching.
So here essentially what … |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 51:
32 | is which is the action expert here which
is a it's based on flow matching and uh is a it's based on flow matching and uh is a it's based on flow matching and uh
the training target that you see here i… |
diffusion-principles-vizuara |
L1: Lecture 10 - Constructing Training Target for Flow Models | Principles of Diffusion Models | 52:
09 | lecture which is going to be like a
magician's reveal. Thank you. |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 00:
03 | Good morning everyone. Good morning everyone.
Today we are starting with the next Today we are starting with the next Today we are starting with the next
lecture of our course which is lecture of our … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 01:
10 | simple which you could have got in the simple which you could have got in the
first lecture itself. first lecture itself. first lecture itself.
But I don't want to show you that route But I don't want… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 02:
06 | keep the mathematics as simple as
possible [snorts] and I have taken possible [snorts] and I have taken possible [snorts] and I have taken
inspiration from an uh MIT course to inspiration from an uh M… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 03:
01 | model is a representation of a phenomena
a neural network is an example of a a neural network is an example of a a neural network is an example of a
model or any mathematical function which model or a… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 04:
22 | we have looked at different techniques
to solve this broader problem. It to solve this broader problem. It to solve this broader problem. It
started out with variational started out with variational s… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 05:
34 | one of the most consequential techniques
in modern [snorts] in modern [snorts] in modern [snorts]
generative modeling. And the reason it generative modeling. And the reason it generative modeling. And… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 06:
33 | and it works exceptionally well. You can
see here they have this action expert see here they have this action expert see here they have this action expert
and this action expert is built [snorts] and … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 07:
26 | formula comes about which is very
simple. So flow modeling at the heart of simple. So flow modeling at the heart of simple. So flow modeling at the heart of
it is very simple and it has huge number it… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 08:
45 | maybe you're sitting on a table right maybe you're sitting on a table right
now or you have a table in front of you. now or you have a table in front of you. now or you have a table in front of you.
U… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 09:
51 | with time. So the magnets that we have
placed on the table, what if these placed on the table, what if these placed on the table, what if these
magnets are constantly moving? magnets are constantly mo… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 10:
59 | take at multiple time steps and hence
you will get a vector field which is you will get a vector field which is you will get a vector field which is
changing with time. So now we are discussing time d… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 12:
12 | depends on not just the position in
space but also it depends on the time space but also it depends on the time space but also it depends on the time
snapshot. So it is a timed dependent snapshot. So … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 13:
07 | timed dependent vector field.
So another way of defining this is So another way of defining this is So another way of defining this is
people usually write this t as a people usually write this t as a… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 14:
11 | So for example, if we take this example
at at the top at time t equal to0, this at at the top at time t equal to0, this at at the top at time t equal to0, this
is what my velocity field looks like. is… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 15:
16 | So now we are asking the question how do
the position of particles which are the position of particles which are the position of particles which are
inside the space change according to inside the spa… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 16:
24 | particle if I know the velocity vector
field. This is exactly what this formula field. This is exactly what this formula field. This is exactly what this formula
says. says. says.
So if I have a parti… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 17:
59 | of x0
okay so now this if I write as delta x okay so now this if I write as delta x okay so now this if I write as delta x
by delta t by delta t by delta t
is equal to mu of u of x0. is equal to mu of… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 19:
08 | box end up at time t equal to 10 let's box end up at time t equal to 10 let's
say. Now if you know the exact velocity say. Now if you know the exact velocity say. Now if you know the exact velocity
at… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 20:
02 | box follows the vector field and the box follows the vector field and the
question I'm trying to address is where question I'm trying to address is where question I'm trying to address is where
does t… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 21:
06 | they represent the vector field which is
time dependent. That is why the arrows time dependent. That is why the arrows time dependent. That is why the arrows
are changing with time. And if we know are… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 22:
21 | but it is not quite correct because at but it is not quite correct because at
the end of it we are doing an the end of it we are doing an the end of it we are doing an
approximation. approximation. ap… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 23:
38 | everywhere in a lot of papers. But flow everywhere in a lot of papers. But flow
means something very specific. In our means something very specific. In our means something very specific. In our
mind, … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 24:
34 | flow is a collection of trajectories
which evolve according to the velocity which evolve according to the velocity which evolve according to the velocity
field. We have just seen that if we know field… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 25:
30 | now let us look at one single snapshot
because we have we have restricted with because we have we have restricted with because we have we have restricted with
the screen in in front of us. Now you the… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 26:
21 | trajectories and we get a overall sense
of the flow. You can see that this this of the flow. You can see that this this of the flow. You can see that this this
is probably something which is swirling.… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 27:
09 | let's let's see how the grid deforms.
Now note that the grid is going to Now note that the grid is going to Now note that the grid is going to
deform in a way that the vector field deform in a way tha… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 28:
09 | according to a trajectory which is
governed by an ordinary differential governed by an ordinary differential governed by an ordinary differential
equation and if we collect all these equation and if w… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 29:
26 | understand how the grid deforms. Yeah.
So this this does work. So this this does work. So this this does work.
Okay. So you can actually change these Okay. So you can actually change these Okay. So yo… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 30:
34 | using the word vector field also. Second
is the trajectories which are calculated is the trajectories which are calculated is the trajectories which are calculated
from the velocity field using the si… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 31:
46 | It is only one dimensional minus theta x
and it is independent of time. So at and it is independent of time. So at and it is independent of time. So at
every time snapshot you take a camera every time… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 32:
52 | is dxt by dt is equal to minus theta of
xt. xt. xt.
Now let's see if this is indeed the Now let's see if this is indeed the Now let's see if this is indeed the
flow. flow.
If we substitute this in xt,… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 34:
06 | is itself completely zero. So the
position stays zero. But position stays zero. But position stays zero. But
let's look at yeah if x0 is any any any let's look at yeah if x0 is any any any let's look … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 35:
12 | example
x0 into e to minus theta t and we get the exact same plot. and we get the exact same plot.
So we can actually play this and see how So we can actually play this and see how So we can actually … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 36:
03 | field and how the flow is a collection
of the trajectories. of the trajectories. of the trajectories.
[snorts] Okay. Now uh once we are getting Okay. Now uh once we are getting
confident with the defi… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 37:
06 | We take small steps in the direction of
the velocity field and we see how the the velocity field and we see how the the velocity field and we see how the
particle trajectory changes with time. particl… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 37:
56 | long is your simulation carried on.
Okay. So we initialize time as t equal Okay. So we initialize time as t equal Okay. So we initialize time as t equal
to0. The step size which is delta t is 1 to0. T… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 38:
56 | position. That is all is mentioned in
the flowchart. So basically the flowchart. So basically the flowchart. So basically
if we have a particle and if we know the if we have a particle and if we know … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 39:
45 | For example, when we are using flow
models for robotics or protein models for robotics or protein models for robotics or protein
synthesis, synthesis, synthesis,
people calculate the flow which is the… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 40:
49 | Now we have defined the word flow in the Now we have defined the word flow in the
initial part of the lecture. We will initial part of the lecture. We will initial part of the lecture. We will
underst… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 41:
41 | So this is exactly what we are going to So this is exactly what we are going to
do in flow models. Also our goal is to do in flow models. Also our goal is to do in flow models. Also our goal is to
con… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 43:
05 | So we use the same ordinary differential So we use the same ordinary differential
equation dx by dt is equal to u. But equation dx by dt is equal to u. But equation dx by dt is equal to u. But
this ti… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 43:
58 | we have added in flow model is that the we have added in flow model is that the
collection of these particles represent collection of these particles represent collection of these particles represent
… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 44:
54 | evolves from t =0 to t = 1. evolves from t =0 to t = 1.
So this is sort of what we want. We want So this is sort of what we want. We want So this is sort of what we want. We want
to start from the ini… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 45:
57 | the start distribution which is a simple
gshian. It is uniformly distributed. gshian. It is uniformly distributed. gshian. It is uniformly distributed.
There are maybe around 2,000 particles There are… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 46:
47 | from the start distribution to the end
distribution. Now here there are a lot of options. We Now here there are a lot of options. We
can convert can convert can convert
we can change the end distribut… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 47:
42 | replaced by a neural network. replaced by a neural network.
So there is another word for this in So there is another word for this in So there is another word for this in
literature which is also call… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 48:
29 | model is being trained and the loss is
going down. What is the meaning of this going down. What is the meaning of this going down. What is the meaning of this
statement? The flow model is going to be … |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 49:
25 | see the animation over here once this
code runs. Yeah. So let's see uh as expected we Yeah. So let's see uh as expected we
start with the initial gshian start with the initial gshian start with the in… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 50:
30 | models we had a number of encoders and a
number of decoders. number of decoders. number of decoders.
So they were discrete. Here something a So they were discrete. Here something a So they were discre… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 52:
03 | the particles who are compressing inside
here they are moving out and here they are moving out and here they are moving out and
slowly resembling that spiral shape. So slowly resembling that spiral sh… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 52:
56 | initial problem of converting the uh initial problem of converting the uh
gshian distribution to our actual gshian distribution to our actual gshian distribution to our actual
distribution. What we sa… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 53:
50 | So the differential equation is also
termed as a neural ordinary differential termed as a neural ordinary differential termed as a neural ordinary differential
equation. So the terms that I have equat… |
diffusion-principles-vizuara |
L2: Lecture 9 - Introduction to Flow Models | Principles of Diffusion Models | 54:
43 | ends our first lecture and in the next ends our first lecture and in the next
lecture we will understand what is the lecture we will understand what is the lecture we will understand what is the
targe… |