cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 00:
03 | Okay, so let's get started. uh so far uh Okay, so let's get started. uh so far uh
our last class we have talked about you our last class we have talked about you our last class we have talked about yo… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 00:
48 | Okay. Uh so this is just like a quick
like reminder of like like reminder of like like reminder of like
what happened next last class. So what happened next last class. So what happened next last clas… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 01:
39 | lastly we talked about like the easiest lastly we talked about like the easiest
way to do discrete diffusion which is way to do discrete diffusion which is way to do discrete diffusion which is
mass d… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 02:
46 | probability path, right? to go from like
some noise or like some the full mass to some noise or like some the full mass to some noise or like some the full mass to
clean data, right? Yeah, basically t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 03:
32 | let let's look at them in a in the
discrete settings. So last time we kind discrete settings. So last time we kind discrete settings. So last time we kind
of skipped through this part uh or of skipped… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 04:
23 | distribution, right? So the probability
the sum of the probability over all the sum of the probability over all the sum of the probability over all
possible states or all possible tokens possible stat… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 05:
13 | transitions should add up to zero. And
this is like and this is literally just this is like and this is literally just this is like and this is literally just
a con conversation uh conservation of a c… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 06:
09 | likely that you're going to go basically
it's like how how how likely that you're it's like how how how likely that you're it's like how how how likely that you're
going to go jump from this state to … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 07:
03 | It's a velocity, right? Essentially. So It's a velocity, right? Essentially. So
it's basically just saying that like um it's basically just saying that like um it's basically just saying that like um
… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 07:
53 | this is also why we're trying to do
reparameterization trick yeah but that's reparameterization trick yeah but that's reparameterization trick yeah but that's
a great question. All right. Anything a g… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 08:
58 | this state that's it right this is like
super uh very intuitive I hope is super uh very intuitive I hope is super uh very intuitive I hope is
All good. What can we observe here? Can we write What can … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 10:
17 | not equal anymore. So just sum of all
the z all the possible z's and then qzx the z all the possible z's and then qzx the z all the possible z's and then qzx
ptz right and z could be equal to x as ptz… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 11:
21 | oiler, right? So you just get you you
just literally plug everything in here just literally plug everything in here just literally plug everything in here
and then you'll get this thing and the and th… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 12:
15 | like the the core of the the story in
all the discrete diffusion papers is all the discrete diffusion papers is all the discrete diffusion papers is
that like no matter what you do try to that like no… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 13:
03 | just use cross entropy to learn this,
right? Because we have the ground truth right? Because we have the ground truth right? Because we have the ground truth
clean data. So we just need to use cross c… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 14:
01 | flips right. So you start from t t flips right. So you start from t t
equals z which and then you sample from equals z which and then you sample from equals z which and then you sample from
your prior… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 14:
45 | you just increment the time and then you
do it over over again until you reach do it over over again until you reach do it over over again until you reach
the end point. Okay, make sense? This this is… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 15:
42 | things that we developed somehow but
basically there are two discreful basically there are two discreful basically there are two discreful
matching papers uh this time they are matching papers uh this… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 16:
22 | know but anyway it's like really like I
really hope this guy get rich and famous really hope this guy get rich and famous really hope this guy get rich and famous
soon. Uh anyway and then the second o… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 17:
05 | of learn the rate matrix but we don't of learn the rate matrix but we don't
actually learn the rate matrix directly actually learn the rate matrix directly actually learn the rate matrix directly
we w… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 17:
50 | your par paragraph is in order to build
this model and this is just like super this model and this is just like super this model and this is just like super
like not good right this is a very very lik… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 18:
36 | like it's the same problem as the fixed
length thing, right? Yeah. length thing, right? Yeah. length thing, right? Yeah.
>> I heard this from like who's doing >> I heard this from like who's doing >> … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 19:
20 | right. So let's take a look at it. Uh right. So let's take a look at it. Uh
yeah. So essentially variable length you yeah. So essentially variable length you yeah. So essentially variable length you
s… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 19:
57 | like something that we can think about,
right? And it's like now we have like right? And it's like now we have like right? And it's like now we have like
two um like frameworks that like sort of two u… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 20:
56 | >> Yeah. What else? >> Yeah. What else?
>> Inserting at but I saw that you can do >> Inserting at but I saw that you can do >> Inserting at but I saw that you can do
decoding in blocks like you can pr… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 21:
29 | >> Okay, so tatting at the end. So you can >> Okay, so tatting at the end. So you can
discard some of the tokens. Yeah, that discard some of the tokens. Yeah, that discard some of the tokens. Yeah, th… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 22:
15 | paragraph and for the next diffusion
process that paragraph is that one process that paragraph is that one process that paragraph is that one
element for you. You noise the paragraph element for you. … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 23:
09 | the same thing and that this is like
this is corresponding to one paper that this is corresponding to one paper that this is corresponding to one paper that
we're going to talk about but not now we're… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 23:
49 | different tokens in between tokens,
right? I don't need to insert at the right? I don't need to insert at the right? I don't need to insert at the
end. Why do I have to insert at the end? end. Why do … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 24:
31 | >> Does this work in the diffusion case?
>> Good question. I I'm gonna talk about >> Good question. I I'm gonna talk about >> Good question. I I'm gonna talk about
it. Okay. So, how do we even model t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 25:
15 | you use it to to to essentially
construct construct the the ground truth construct construct the the ground truth construct construct the the ground truth
of like what this like what the full of like … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 25:
57 | right? What is this other thing?
>> The position or the whether to generate >> The position or the whether to generate >> The position or the whether to generate
it between these it between these it b… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 26:
52 | basically if we're missing tokens and basically if we're missing tokens and
like essentially like essentially like essentially
note that like it's more likely or like note that like it's more likely o… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 27:
45 | what you're predicting and then the
second objective from your model is second objective from your model is second objective from your model is
predict what tokens are missing so this predict what tok… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 28:
26 | like I don't know.
>> Yeah. >> Yeah.
>> Yeah. But basically what Okay. So >> Yeah. But basically what Okay. So >> Yeah. But basically what Okay. So
basically what happens in the model or basically wha… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 29:
10 | >> Yeah. Yeah. So basically what's
happening in what's hap what's happening happening in what's hap what's happening happening in what's hap what's happening
in like implementation is that you are in … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 29:
55 | is that is missing out of all the tokens is that is missing out of all the tokens
that are missing? Right. Okay. So these that are missing? Right. Okay. So these that are missing? Right. Okay. So thes… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 30:
48 | missing versus you know how many tokens
you predict to be missing and then the you predict to be missing and then the you predict to be missing and then the
second thing is uh like literally just seco… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 31:
48 | flow
And why is it a flow model? Well, it's And why is it a flow model? Well, it's And why is it a flow model? Well, it's
actually because this thing like the actually because this thing like the actu… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 32:
35 | another thing then you should be doing another thing then you should be doing
this operation where it's a ratio or this operation where it's a ratio or this operation where it's a ratio or
like a rate… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 33:
23 | divergence that is basically just a
generalized version of KL. Uh but this generalized version of KL. Uh but this generalized version of KL. Uh but this
is like too complicated. We're not going is lik… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 34:
11 | actually like already good enough for
whatever you want to do. So you can whatever you want to do. So you can whatever you want to do. So you can
literally just train a insertion based literally just … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 34:
56 | we kind of fixed a veryable
length uh problem with insertion based length uh problem with insertion based length uh problem with insertion based
generation. Uh we have other things to generation. Uh w… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 35:
40 | So like basically this thing is very
nice because you can um like essentially nice because you can um like essentially nice because you can um like essentially
you can do paralyze generation like you … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 36:
20 | that you have generated like in your
previous blocks. The only thing you need previous blocks. The only thing you need previous blocks. The only thing you need
to like do birectional attention to is t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 37:
00 | people think about this. Um but yeah, so people think about this. Um but yeah, so
this is like a nice way to do you know this is like a nice way to do you know this is like a nice way to do you know
t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 37:
50 | >> not in the current block but but we can
K catch all the previous blocks. >> No because we don't we haven't generated >> No because we don't we haven't generated
that yet, right? So like the block a… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 38:
29 | of like also like but but but but if you
have too small of a blog then it just have too small of a blog then it just have too small of a blog then it just
Yeah. It doesn't really it just is just Yeah.… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 39:
16 | in I think. Yeah. in I think. Yeah.
>> Do you think the second one is >> Do you think the second one is >> Do you think the second one is
um like it's it's the it's a problem of um like it's it's the … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 40:
11 | That's correct. Yeah.
>> For the for the second case, the mass >> For the for the second case, the mass >> For the for the second case, the mass
diffusion, why can't we do something diffusion, why can… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 40:
57 | Anything else? Yep.
>> Can you overhand why mass diffusion is >> Can you overhand why mass diffusion is >> Can you overhand why mass diffusion is
lower? lower? lower?
>> Uh okay. Who wants to tell who… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 41:
49 | but it doesn't mean that you you cannot
train a good enough diffusion model to train a good enough diffusion model to train a good enough diffusion model to
blm that's not the case it's just that blm … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 42:
45 | the the optimal that you're going to get the the optimal that you're going to get
from whatever algorithm that they from whatever algorithm that they from whatever algorithm that they
develop will loo… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 43:
32 | this is not image and text but like
we're going to look at image and text we're going to look at image and text we're going to look at image and text
later. But for those of you who are in later. But … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 44:
31 | >> What are we cogenerating?
>> So, we're co-generating something uh >> So, we're co-generating something uh >> So, we're co-generating something uh
discreet and something uh continuous at discreet an… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 45:
15 | like alternating between the two that's like alternating between the two that's
pretty much that right and like pretty much that right and like pretty much that right and like
basically And then if yo… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 46:
03 | well right so what you can do is you can well right so what you can do is you can
uh have like just like I think some uh have like just like I think some uh have like just like I think some
someone al… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 46:
50 | >> I mean the problem is like this is a >> I mean the problem is like this is a
research project right so it's like very research project right so it's like very research project right so it's like ve… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 47:
37 | and then the the the image for example
like flower or like cat and then and like flower or like cat and then and like flower or like cat and then and
then the the the the image are going to then the t… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 48:
13 | text and like image at the same time text and like image at the same time
because like you start from nothing and because like you start from nothing and because like you start from nothing and
then y… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 48:
49 | the way to become omni but this is like
just like two two modalities I guess. just like two two modalities I guess. just like two two modalities I guess.
Yeah. Yeah. Omni model just means that Yeah. Y… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 49:
50 | you cannot access the ponapto anymore,
it's still going to be on YouTube. Uh I it's still going to be on YouTube. Uh I it's still going to be on YouTube. Uh I
kind of do some editing uh for the kind o… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 50:
25 | your turn to take what you've learned
into the real world so I will highly into the real world so I will highly into the real world so I will highly
recommend if you are proud of what recommend if you… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 51:
05 | something like that. I think it could be something like that. I think it could be
a good experience if you guys are a good experience if you guys are a good experience if you guys are
interested in ge… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 51:
39 | research slash you know when you go to
the AGI company you know make the AGI a the AGI company you know make the AGI a the AGI company you know make the AGI a
diffusion model please we don't want to d… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 52:
25 | >> padding tokens and tokens after the OS,
right? right?
>> Yeah. >> Yeah.
>> Uh so one thing I implemented for >> Uh so one thing I implemented for >> Uh so one thing I implemented for
homework 3 and… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 53:
06 | I thought that's what they did in I thought that's what they did in
refflow but maybe not. If it's not then refflow but maybe not. If it's not then refflow but maybe not. If it's not then
this is a go… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 53:
42 | than a 2 RF model because the only than a 2 RF model because the only
errors accumulate. errors accumulate. errors accumulate.
>> Yeah. No, that's your homework. Four. >> Yeah. No, that's your homewor… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 54:
19 | in general I think that's a better idea
than the normal the the the way that than the normal the the the way that than the normal the the the way that
they they did. Okay. Any other question? they the… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 55:
03 | >> Yeah. Yeah. So actually there multiple >> Yeah. Yeah. So actually there multiple
uh so the most famous one in the uh so the most famous one in the uh so the most famous one in the
robotics world is… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 55:
49 | gradient you should apply from those
buffer data to your current time to one buffer data to your current time to one buffer data to your current time to one
of the time that you sampled. Um so this of… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 56:
25 | in that framework but you can turn you in that framework but you can turn you
can essentially turn an S OD like a flow can essentially turn an S OD like a flow can essentially turn an S OD like a flow… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 57:
06 | basically is there any other ways that
we can do it besides turning into a SDE we can do it besides turning into a SDE we can do it besides turning into a SDE
um yeah so like let's hope we can get um … |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 57:
58 | essentially you just sort of like uh
they sort of extend the flow map uh uh they sort of extend the flow map uh uh they sort of extend the flow map uh uh
what you call framework into the what you call… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 58:
37 | Uh, so let's see if he actually post it
online this week. Uh, but yeah, I don't online this week. Uh, but yeah, I don't online this week. Uh, but yeah, I don't
actually know how how he did it, but act… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 59:
19 | of similar to uh one of the other
previous uh framework and equilibrium previous uh framework and equilibrium previous uh framework and equilibrium
model. So equilibrium model is something model. So e… |
cmu-10799-diffusion-flow |
L12: CMU 10799 S26: Lecture 13 - Discrete Flow Matching & Edit Flow - Diffusion & Flow Matching | 60:
02 | or the underlying uh, you know, uh, math
is a little bit related, but like this is a little bit related, but like this is a little bit related, but like this
is definitely something new. I think is de… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 00:
04 | Okay. So, previously uh we have learned Okay. So, previously uh we have learned
about fast sampling techniques, right? about fast sampling techniques, right? about fast sampling techniques, right?
And… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 00:
59 | pre-trained model, right? So suppose we
already have a pre-trained diffusion already have a pre-trained diffusion already have a pre-trained diffusion
model like before. Um then what is the model like… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 02:
00 | second time step we do a loss comparison
with the 500 time step. Hey yes very with the 500 time step. Hey yes very with the 500 time step. Hey yes very
very good very good very very good very good ver… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 02:
45 | t and uh how about okay let's just do a
few more steps right how about we'll few more steps right how about we'll few more steps right how about we'll
train a new model such that um like one train a n… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 03:
44 | A So this is like yep
>> exact as >> exact as >> exact as
many as you want essentially. So like many as you want essentially. So like many as you want essentially. So like
log to t I guess. log to t I… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 04:
30 | Right. Right.
>> So if in the first example we took the >> So if in the first example we took the >> So if in the first example we took the
two two two
steps to V1 of my new models step and in steps t… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 05:
11 | way this is the paper that tells you
that you should do V prediction like V that you should do V prediction like V that you should do V prediction like V
prediction is the best parameterization predic… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 05:
59 | >> Oh, why wouldn't it be a good model? I >> Oh, why wouldn't it be a good model? I
guess guess guess
Marco, Marco, Marco,
>> you still have to learn the same amount >> you still have to learn the sam… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 06:
44 | network's job to just take this teeny
tiny Gaussian to that complex version tiny Gaussian to that complex version tiny Gaussian to that complex version
without getting enough. So it's very without get… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 07:
23 | what about like the feasibility of
training this kind of model like how training this kind of model like how training this kind of model like how
about let's just look at uh training a about let's jus… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 08:
17 | to take a very long time and it's like
if you try to scale it up to t time if you try to scale it up to t time if you try to scale it up to t time
steps, right? Uh let me finish maybe. Um steps, right… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 08:
54 | sounds kind of not great. Yeah hold on.
Yeah, Yeah, Yeah,
>> I think the four power could only be for >> I think the four power could only be for >> I think the four power could only be for
the teache… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 09:
31 | just going to become very slow.
So, so do you think that like five total So, so do you think that like five total So, so do you think that like five total
four of the teacher student four of the teach… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 10:
18 | here
>> um and what was the question >> um and what was the question >> um and what was the question
step. step. step.
>> Yeah. >> Yeah.
>> You said you require like four forward >> You said you requi… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 11:
02 | That's your forward pass. That's your forward pass.
>> Yeah but just create a new data set with >> Yeah but just create a new data set with >> Yeah but just create a new data set with
this feature mod… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 11:
37 | yeah. Um, so what do we think? Is it
possible to train a model that goes from possible to train a model that goes from possible to train a model that goes from
t directly xt directly to x0? What do we… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 12:
30 | right even even if you stop red over
there it's not good right okay so anyway there it's not good right okay so anyway there it's not good right okay so anyway
let's take a look at this uh problem let… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 13:
15 | yeah, what does it mean is that
basically for any two points like any basically for any two points like any basically for any two points like any
arbitrary two points on the same OD arbitrary two poin… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 14:
26 | is this the only criteria that we need
for this new model to generate something for this new model to generate something for this new model to generate something
to to generate the correct thing? to t… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 15:
13 | like there it says nothing about the
output, right? Like we have no guarantee output, right? Like we have no guarantee output, right? Like we have no guarantee
about what the output should look like a… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 16:
03 | condition essentially so this is the condition essentially so this is the
thing that's going to like basically thing that's going to like basically thing that's going to like basically
just like make … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 16:
48 | B or something. It's like the when when B or something. It's like the when when
your gradient of the the loss spikes up. your gradient of the the loss spikes up. your gradient of the the loss spikes u… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 17:
42 | consistency model he was at open eye and
then now he's apparently some big name then now he's apparently some big name then now he's apparently some big name
in meta super intelligence lab or in meta … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 18:
26 | DDPM. Yeah. Yeah. So, that guy is also a DDPM. Yeah. Yeah. So, that guy is also a
genius and he's now has his own startup. genius and he's now has his own startup. genius and he's now has his own star… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 19:
15 | green one which is like only one step
away is the one that we're going to use away is the one that we're going to use away is the one that we're going to use
here. So essentially basically what's here… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 20:
01 | next time step that is getting on the
same trajectory solved by our solver. same trajectory solved by our solver. same trajectory solved by our solver.
and the original teacher model. So this and the … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 20:
47 | just going to become a little bit
messier if you if you do not use L2 but messier if you if you do not use L2 but messier if you if you do not use L2 but
in practice in practice in practice
uh percept… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 21:
41 | plus delta t? plus delta t?
>> Ah, that's also another great question. >> Ah, that's also another great question. >> Ah, that's also another great question.
So, uh here everything so this f theta So, … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 22:
24 | that you need to take from the t-shirt
doesn't scale with t. Yeah. Uh, okay. Do doesn't scale with t. Yeah. Uh, okay. Do doesn't scale with t. Yeah. Uh, okay. Do
you want your first you want your firs… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 23:
13 | with the larger jump. We do not update with the larger jump. We do not update
our model with the with the shorter our model with the with the shorter our model with the with the shorter
jump. jump. ju… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 24:
31 | parameterization that we're using sort
of and uh so we are trying to predict of and uh so we are trying to predict of and uh so we are trying to predict
the clean data and um usually it the clean data… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 25:
39 | when t is zero the skip connection is when t is zero the skip connection is
one and then the output the weight is one and then the output the weight is one and then the output the weight is
zero right… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 26:
26 | can be producing garbage. Right now all
this saying is that if my model sees a this saying is that if my model sees a this saying is that if my model sees a
very good image already and also is the ver… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 27:
09 | think I don't I I agree with the thing
that like the loss function on the last that like the loss function on the last that like the loss function on the last
line can if you just do training with lin… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 27:
53 | should not because if we just ask it to
do the same thing at S and T that same do the same thing at S and T that same do the same thing at S and T that same
thing could just be end up predicting thing… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 28:
37 | this. Sorry. The the skip connection
weight is kind of like a schedule. It weight is kind of like a schedule. It weight is kind of like a schedule. It
has a scheduling. It's only one when has a schedu… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 29:
33 | trajectory that's like kind of similar trajectory that's like kind of similar
to what we what we would have got from a to what we what we would have got from a to what we what we would have got from a… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 30:
25 | scale differently essentially. So that scale differently essentially. So that
so that it's like slightly less noisy. so that it's like slightly less noisy. so that it's like slightly less noisy.
Uh th… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 31:
20 | about the previous uh slide, but
>> uh so if we if we already know that the >> uh so if we if we already know that the >> uh so if we if we already know that the
image is close enough to x0 where it's… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 31:
59 | boundary condition. Yeah. Okay. All
right. Any any questions about training right. Any any questions about training right. Any any questions about training
consistency model from scratch? consistency … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 32:
50 | Um and we're not trying to solve it
here. Well we kind of are trying to here. Well we kind of are trying to here. Well we kind of are trying to
solve it here actually. Um solve it here actually. Um so… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 33:
47 | So one step but how do you do multi-step
right? Can you even do multi-step? right? Can you even do multi-step? right? Can you even do multi-step?
>> Yeah like we're turning on a lot of like >> Yeah li… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 34:
20 | >> Why do you want to do that's a great
question. The next slide is going to question. The next slide is going to question. The next slide is going to
answer that. Yeah. answer that. Yeah. answer that… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 35:
04 | or compute that you can afford to do
four step generation say but like this four step generation say but like this four step generation say but like this
model is not trained for that it doesn't model… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 35:
44 | do whatever I guess. Yeah, doesn't do whatever I guess. Yeah, doesn't
really matter anymore. Um yeah so really matter anymore. Um yeah so really matter anymore. Um yeah so
essentially essentially esse… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 36:
50 | >> instead of doing like this one step to >> instead of doing like this one step to
the final noise you can do it like two the final noise you can do it like two the final noise you can do it like two… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 37:
27 | be able to jump to any point from any
point to any point. Um so yeah so this point to any point. Um so yeah so this point to any point. Um so yeah so this
xt is also arbitrary so you should be xt is a… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 38:
20 | >> Like two points on the trajectory then
you saw the you saw the OD between them you saw the you saw the OD between them you saw the you saw the OD between them
and then you essentially just take the… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 39:
04 | meaningful when you're dealing with
clean data. So think about it right the clean data. So think about it right the clean data. So think about it right the
perceptual loss is like something like a per… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 39:
52 | perceptual distance between the final X0 perceptual distance between the final X0
output. Um the only thing that you need output. Um the only thing that you need output. Um the only thing that you nee… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 40:
30 | >> Yeah. Yeah. So like basically so the
this the first part uses solvers so that this the first part uses solvers so that this the first part uses solvers so that
this thing this part is actually on t… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 41:
18 | im style at at noise bag thing right and
and that's still valid actually in the and that's still valid actually in the and that's still valid actually in the
in the paper like we have this thing in th… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 42:
02 | kicked off that
>> uh what do you mean? Oh you mean like >> uh what do you mean? Oh you mean like >> uh what do you mean? Oh you mean like
the the Oh yeah. So so like you mean the the Oh yeah. So so l… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 42:
59 | Yeah. But basically because we do not Yeah. But basically because we do not
always match the end point now we we always match the end point now we we always match the end point now we we
instead of a … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 43:
52 | score right from jumping from T. Now um
you you get your exact log likelihood you you get your exact log likelihood you you get your exact log likelihood
back by because you can take the back by becau… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 44:
43 | change that you're going from a that
you're going from any point should you're going from any point should you're going from any point should
follow the instant like the tangent line follow the instan… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 45:
30 | when S is not equal to T and then score when S is not equal to T and then score
matching loss which is just go from T to matching loss which is just go from T to matching loss which is just go from T … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 46:
13 | >> Uh which one? Uh you're saying that we >> Uh which one? Uh you're saying that we
add like a discriminator to improve FID add like a discriminator to improve FID add like a discriminator to improve … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 47:
17 | otherwise it doesn't make sense I guess.
Yeah. >> Yeah. Oh sorry. Yeah. >> Yeah. Oh sorry. Yeah.
>> How do you weigh the different loss >> How do you weigh the different loss >> How do you weigh the d… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 48:
24 | so I guess someone asked right do Do you so I guess someone asked right do Do you
even need a solver here? Uh the answer even need a solver here? Uh the answer even need a solver here? Uh the answer
i… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 49:
20 | Is it just like all the questions are
like, "Oh, the thing that we're going to like, "Oh, the thing that we're going to like, "Oh, the thing that we're going to
talk about next." Very nice. Okay. talk… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 50:
04 | OD or PD or whatever um and then the
flow maps because we also look at ODEs flow maps because we also look at ODEs flow maps because we also look at ODEs
right especially if If you're doing flow right… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 51:
03 | It's very sad. Anyway, um but how do you
uh build a valid flow map? Actually the uh build a valid flow map? Actually the uh build a valid flow map? Actually the
answer to this question uh can be found… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 51:
49 | prediction of XS. Okay. So flow map is prediction of XS. Okay. So flow map is
the predict prediction of XS the the predict prediction of XS the the predict prediction of XS the
endpoint of the jump. O… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 52:
30 | guess. Uh let's just like take this
analogy. Okay. Yes. analogy. Okay. Yes. analogy. Okay. Yes.
>> Call it progressive. Is that the same >> Call it progressive. Is that the same >> Call it progressive… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 53:
21 | that is flowing down this one river. I that is flowing down this one river. I
can teleport the duck so that the duck's can teleport the duck so that the duck's can teleport the duck so that the duck's… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 54:
00 | get like basically okay. So I guess this
is a better uh uh visualization. So this is a better uh uh visualization. So this is a better uh uh visualization. So this
is generated by nano banana the thin… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 54:
39 | check for you. And then and then the the
cop be like, huh, let's see if there's cop be like, huh, let's see if there's cop be like, huh, let's see if there's
any duck that is like suspicious. Like, an… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 55:
24 | instantaneous jump so that you can fold
the cop and then and then be like, "Oh, the cop and then and then be like, "Oh, the cop and then and then be like, "Oh,
okay. This is a normal duck." Basically,… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 56:
25 | >> Yes.
>> So if you take a langian from S to S >> So if you take a langian from S to S >> So if you take a langian from S to S
plus DT, it should be the same as the is plus DT, it should be the same … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 57:
37 | well for consistency model we don't
really match the the the derivative. I really match the the the derivative. I really match the the the derivative. I
guess that's the problem. guess that's the prob… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 58:
33 | I can do I can be a little bit like I
can essentially be a a little bit um can essentially be a a little bit um can essentially be a a little bit um
like late. I can I I can start the jump like late. … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 59:
57 | jumping, they should end up at the same jumping, they should end up at the same
place. So, as long as they're following place. So, as long as they're following place. So, as long as they're following
… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 60:
59 | really mention the flow map thing in really mention the flow map thing in
their in their paper. We by we I mean their in their paper. We by we I mean their in their paper. We by we I mean
one of my pa… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 61:
43 | displacement that you're predicting
right so this is like so the right so this is like so the right so this is like so the
displacement that you're predicting is displacement that you're predicting is… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 62:
36 | they call a mean flow identity. they call a mean flow identity.
Okay, that does it make sense for people? that does it make sense for people?
Good. Okay. So why is mean flow a oiler Good. Okay. So why… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 63:
31 | that's it. Um and one nice thing about
the mean flow identity is that uh so in the mean flow identity is that uh so in the mean flow identity is that uh so in
the middle here if you take s equals t th… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 64:
47 | like but an interesting thing is that
CTN was developed in 2023 CTN was developed in 2023 CTN was developed in 2023
and um the the how to train your like and um the the how to train your like and um t… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 65:
43 | think this is the most intuitive one. So
this is literally just saying that uh this is literally just saying that uh this is literally just saying that uh
basically if your flow map jump from S basica… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 66:
44 | How many any of you guys have seen this
model? Shortcut model. No one. No. I model? Shortcut model. No one. No. I model? Shortcut model. No one. No. I
think this is a Peter Ail or something. think thi… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 67:
33 | making sure that your flow oh my god I'm
so sorry we're running out of time but so sorry we're running out of time but so sorry we're running out of time but
let's get let's finish everything uh the l… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 68:
31 | likelihood. Uh so why why is that? Well,
it's basically because yeah because it's it's basically because yeah because it's it's basically because yeah because it's
only jump to in the sampling space a… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 69:
13 | can use the same algorithms to solve
another OD right so this is what we're another OD right so this is what we're another OD right so this is what we're
doing so this is like my new paper doing so th… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 69:
53 | use the same full map method to distill
any OD. So if you have like you know an any OD. So if you have like you know an any OD. So if you have like you know an
interesting OD that is associated with i… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 70:
40 | He was a PhD student at Stanford. Uh he He was a PhD student at Stanford. Uh he
co-founded something and then got co-founded something and then got co-founded something and then got
acquired by Luma. … |