概念

Latent Diffusion

— 先用 VAE 把图压成小张量,再让 diffusion 在小张量空间里跑。

很多人第一次看到 Stable Diffusion 论文标题就以为:latent diffusion = 把 VAE 拼到 diffusion 前面,端到端一起训。这是最常见的误解。真相是:latent diffusion 是 TWO-STAGE 训练,VAE 提前完成了「感知压缩」(perceptual compression),然后才轮到 diffusion 上场。两阶段的 loss、数据、目标都不一样,中间用 E.requires_grad_(False) 把 encoder/decoder 冻得死死的。如果你把它当成一阶段端到端模型去推梯度公式,会发现根本对不上——因为 diffusion 训练时 VAE 完全是个常量函数。

打个上课用得到的比喻:VAE 像后台美术,diffusion 像剧作家。后台美术先把"512×512 像素的画布"折叠成"64×64×4 的小笺"——这小笺保留了所有能被人眼分辨的语义,丢掉了人眼根本看不出区别的高频抖动(一根头发丝的具体灰度值)。剧作家拿到小笺之后,只用专心写"该出现什么内容",不用操心"每根头发怎么渲染"。最后把写好的小笺递给后台美术,由 decoder 一次性铺成 512×512 的成品。这就是为什么 Rombach 把第一阶段叫做 perceptual compression、第二阶段叫做 semantic compression:前者削掉"看不出来的",后者建模"语义层面"。

另一个等价的比喻是地图测绘:你要在地图上画一条河流的走势,没必要把每滴水的位置都建模——把河面抽象成几个控制点就够了。VAE encoder 干的就是"把河面抽象成控制点",diffusion 在控制点空间里规划走势,最后 decoder 把控制点还原成连续河面。计算量集中在抽象层,效率自然高。

「过去五年,diffusion 和 flow matching 主宰了视觉生成 AI 的版图,催生了极强的 text-to-image、text-to-video 模型——GPT-4o image、nano banana、Midjourney、Flux……背后的核心技术就是 diffusion / flow matching。」—— CMU 10799 L11, Linqi (Alex) Zhou (Luma AI), @00:44
「Luma 自己也训了大规模 text-to-video 模型,主打真实感……可以看到这些模型在生成运动和真实角色动画上非常擅长。」—— CMU 10799 L11, @01:36(Alex 在介绍 Luma R3 视频生成模型时)

讲师 Alex 在 CMU L11 没有专门展开 latent diffusion 这一节,但他列举的所有商用图/视频模型(GPT-4o image、nano banana、Midjourney、Flux、Luma R3)全部跑在 latent space 里。换句话说:你在 2026 年看到的几乎所有"能跑得动 4K 视频"的生成模型,都靠 latent diffusion 这一招把计算量压下来——否则 GPU 集群烧不起。Luma 的 R3 视频模型 latent 是 3D(含时间维),但配方完全沿用 Rombach 2022 的两阶段框架。

训练里这件事对应:阶段一把 $E, D$ 训成一个能保留语义的"图像压缩器";阶段二把 U-Net 训成一个"在压缩域里去噪"的网络。后者才是大家熟悉的 $\epsilon$-prediction,只是 $x$ 换成了 $z$。再往后,无论是 classifier-free guidance、DDIM、DPM-Solver 还是 flow matching,所有采样技巧无一例外地继续在 latent 空间起作用——只要把训练阶段的 token 维度认成 $z$ 而不是 $x$,整套数学不动一行。这是 latent diffusion 范式的真正威力:它把"压缩"和"生成"解耦,让后续所有进展都自动继承速度红利。

从另一个角度看:CMU L11 的 Alex 在白板上画的 OD trajectory(probability flow ODE 的蓝色路径)本来是在数据分布和高斯先验之间连线;latent diffusion 把这两端的"数据分布"换成了"latent 分布",trajectory 的形状和数学不变,只是脚下的空间从百万维像素流形变成了万级 latent 流形。这种"换空间不换数学"的设计思路在深度学习里非常少见——绝大多数加速技巧都要付出"重新推一遍 loss"的代价,而 latent diffusion 几乎是免费的。代价只有一个:你需要先训好那个 VAE。

两阶段是哪两阶段?

Rombach et al. 2022《High-Resolution Image Synthesis with Latent Diffusion Models》给出的核心配方就是把生成任务拆成两段独立训练。论文的关键洞察是:在像素空间直接跑 diffusion,模型实际上在做两件不同的事情——既要学"全局语义"(什么物体出现在哪里),又要学"高频细节渲染"(每根头发的灰度抖动)。这两件事的难度、所需建模能力、数据效率完全不同,硬塞给同一个 U-Net 就是事倍功半。Rombach 的解法:用一个生成 GAN 风味的 VAE 把高频渲染部分预先学完、固化在 decoder 里;剩下的语义建模交给 latent 空间的 diffusion。

Stage 1: Perceptual Compression(训 VAE)

输入 $x \in \mathbb{R}^{H \times W \times 3}$(比如 $512 \times 512 \times 3$,共 786 432 维)。Why 不直接在像素跑 diffusion:像素空间里一大半比特用在描述"人眼根本分辨不出"的高频细节上——直接在像素跑 diffusion 等于让模型把算力浪费在不该建模的层级。Rombach 在论文 Figure 2 里画了一张著名图:横轴 distortion(PSNR),纵轴 rate(bits);曲线上"感知失真和率失真分道扬镳"的那个拐点就是 perceptual compression 的甜区——压到该拐点为止,人眼分辨不出;压过头,开始丢语义。VAE 的工作就是把模型拽到这个甜区。

训练目标三项相加:

$$\mathcal{L}_{VAE} = \underbrace{\|x - D(E(x))\|_1}_{\text{重建}} + \underbrace{\lambda_{KL} \cdot D_{KL}(q(z|x) \| \mathcal{N}(0, I))}_{\text{正则}} + \underbrace{\lambda_{adv} \cdot \mathcal{L}_{GAN}(x, D(E(x)))}_{\text{对抗}}$$

Why 每一项:重建(L1 比 L2 更鲁棒、对 outlier 不敏感)保证 decoder 能还原;KL 保证 latent 分布不至于退化成离散点(但权重极小,$\lambda_{KL} \approx 10^{-6}$,否则模糊);对抗损失用 PatchGAN 判别器保证 decoder 输出"看起来真实"——单靠 L1 重建会糊,加 adversarial 之后纹理才锐。Why 不直接用大 KL 权重:那就退化成纯 VAE,重建模糊到 diffusion 都救不回;Stable Diffusion 的 KL 系数几乎可以忽略,本质上更像 "regularized AE + GAN",名字叫 VAE 只是历史包袱。

输出 latent $z \in \mathbb{R}^{64 \times 64 \times 4}$(共 16 384 维)。压缩比 = $786{,}432 / 16{,}384 = 48\times$。Stable Diffusion 用 $f=8$(每维空间下采 8 倍),channel 数 = 4。Why $f=8$ 而不是更大:Rombach 实验扫了 $f \in \{4, 8, 16, 32\}$,$f=4$ FID 最好但算力收益不够,$f=16$ 起重建质量崩,$f=8$ 是"FID 与速度的帕累托最优"。

工程提示:Stable Diffusion 公开发布的 VAE checkpoint sd-vae-ft-mse 是社区在原版基础上用 MSE loss + 更大数据集 finetune 出来的,比原版纹理更稳定,跑文生图前默认换掉。另一个常踩的坑是 fp16 推理时 VAE encoder 会数值溢出(latent 出现 NaN),需要在 encoder 部分强制 fp32 跑、只把 U-Net 跑 fp16,这是 diffusers 库里默认的 upcast_vae 行为。

Stage 2: Latent Diffusion(冻 VAE,训 U-Net)

冻住 $E, D$。对每个训练样本 $x_0$:

  1. $z_0 = E(x_0)$,得到 $64 \times 64 \times 4$ 的"干净 latent";
  2. 乘以 scale_factor=0.18215 把 latent 重标定到接近单位方差(关键工程细节,见 pitfall #3);
  3. 采 $t \sim \mathcal{U}\{1, \ldots, T\}$、$\epsilon \sim \mathcal{N}(0, I)$;
  4. 加噪:$z_t = \sqrt{\bar\alpha_t}\, z_0 + \sqrt{1-\bar\alpha_t}\, \epsilon$;
  5. U-Net 预测 $\hat\epsilon = \epsilon_\theta(z_t, t, c)$,其中 $c$ 是 text embedding;
  6. loss = $\| \epsilon - \hat\epsilon\|^2$(Ho 2020 的 simple loss,未带任何重加权)。

Why $\epsilon$-prediction 在 latent 空间依旧成立:noise schedule、ELBO、Tweedie 公式全部对 $z$ 同样推导——只要 $z$ 维度有限、近似高斯,DDPM 的所有数学不需要改一行。这就是为什么 Rombach 论文里能直接复用 Ho 2020 的 simple loss,整篇论文几乎没有新的 diffusion 数学,全部贡献集中在"换个空间跑"。

引用 CMU L11 Alex 对 simple loss 的总结:「最终损失归结为一个简单的 L2 loss,极其稳定,对大参数量也极易 scale。」(@04:13)这句话在 latent diffusion 上同样成立,且因为 latent 维度更低,每步反向传播的显存占用骤降,反而让大模型更容易训。

两阶段的训练时长与数据

Stage 1 训练 VAE 用 OpenImages 数据集,约 250 万张图,A100 单卡训 7 天即可收敛——因为 perceptual compression 任务相对独立,不需要文本对齐数据。Stage 2 训 U-Net 用 LAION-5B 子集(约 6 亿张图文对),256 张 A100 训约 25 天得到 SD-1.4。Why 数据量差距这么大:阶段一只需"学会压缩任意自然图像",阶段二要"学会把所有人类语言概念映射到视觉 latent",后者是开放词汇问题,数据量需求高三个数量级。

这套数据分工还有一个隐含的可复用性:阶段一训出的 VAE 可以被多个 diffusion 模型共享——SD-1.x、SD-2.x、ControlNet、各种 finetune 用的都是同一个 kl-f8 VAE checkpoint。社区每年消耗的"训 VAE"算力其实只在 2022 年发生过一次。这种"前置压缩器一次训好,万人共享"的范式后来也启发了语音生成(EnCodec 之于 AudioLM)、3D 生成(DINO feature 之于 3D diffusion)等多个领域,是 latent diffusion 思想最深远的间接影响。

条件输入:cross-attention

U-Net 的每个 transformer block 里加 cross-attention:query 来自 latent feature map,key/value 来自 text encoder(早期用 CLIP ViT-L,SD3 用 T5-XXL + CLIP-L + CLIP-G 三合一):

$$\text{Attn}(Q, K, V) = \text{softmax}\!\left(\frac{Q K^\top}{\sqrt{d}}\right) V, \quad Q = W_Q \phi(z_t),\ K = W_K \tau_\theta(c),\ V = W_V \tau_\theta(c)$$

这里 $\phi(z_t)$ 是 U-Net 中间层的 spatial feature(把 $H \times W$ 拍平成 token 序列),$\tau_\theta(c)$ 是 text encoder 的输出(77 个 token,每个 token 768 维)。每个 spatial token "去问" 77 个 text token "你和我有多相关",再加权聚合到自己身上。

Why 用 cross-attention 而不是 concat:text token 数和 spatial token 数不一样(77 vs $64{\times}64$),concat 会强制 padding;cross-attention 允许任意条件长度,还能让模型显式学到"哪个词关注哪块区域"——可视化 attention map 能看到"dog"这个词的注意力聚焦在图里狗的轮廓上,这是 latent diffusion 可解释性研究(Prompt-to-Prompt、Attend-and-Excite)的基石。

Why 注入在 U-Net 中间层而不是输入层:U-Net bottleneck 的 feature 已经是抽象语义,比浅层像素更适合和"语义文本"对齐;Rombach 实测在浅层注入 cross-attn 收敛慢、且文图对齐差。

为什么快 50 倍?

U-Net 的算力主要花在 self-attention 上,复杂度 $O(N^2 d)$,$N$ 是空间 token 数。Pixel diffusion:$N = 512 \times 512 = 262{,}144$;latent diffusion:$N = 64 \times 64 = 4{,}096$。比值 $(262144 / 4096)^2 \approx 4096\times$ 的注意力开销缩减,整体(含卷积部分被稀释)实测 ~50× 加速 + ~10× 显存降低。这就是 Stable Diffusion 能在单张消费级 GPU 推理的根本原因。

对比 Imagen(Google 2022):Imagen 在 64×64 pixel 上跑 base diffusion,再用两个 super-resolution diffusion 阶梯放大到 1024×1024,三个模型加起来约 8B 参数。Stable Diffusion 用一个 U-Net (~860M) + 一个 VAE (~80M),单卡 8GB 显存就能跑。前者贵在算力,后者贵在"想到要解耦压缩与生成"——一次架构选择换来一整代消费级生成 AI。值得注意的是:Imagen 走 cascade 路线本质上也在"降维"——它先在低分辨率像素上完成大部分语义建模,再用 SR 阶梯补细节;只不过 cascade 用分辨率降维,latent 用学习到的语义空间降维。后者更彻底,因为 VAE 学到的压缩比固定分辨率下采更聪明:64×64 latent 携带的信息相当于 256×256 像素,但只占 64×64 的算力。

采样与推理

推理流程是训练的镜像:

  1. 从先验采 $z_T \sim \mathcal{N}(0, I)$(这是 64×64×4 的高斯噪声 tensor,不是 512×512×3 的);
  2. 跑 $T'$ 步反向(DDIM 通常取 $T' = 50$,DPM-Solver 可降至 20):每步用 U-Net 预测 $\hat\epsilon$,按采样器更新 $z_{t-1}$;
  3. 得到 $z_0$,乘以 $1/\text{scale\_factor}$ 反归一化;
  4. $x_0 = D(z_0)$ 一次性 decode 出 512×512×3 像素。

注意 decoder 只跑一次——它不参与反复迭代。这意味着即使 decoder 比较慢,分摊到每张图上也几乎可以忽略。如果你以为 decoder 也是迭代去噪结构,那就把 latent diffusion 和 cascade diffusion(Imagen 那种)搞混了:cascade 是多个 diffusion 串联,每一级都迭代;latent diffusion 是一个 diffusion + 一次性 decoder。两者解决"算力"的思路完全不同——cascade 是"分辨率分级摊销",latent 是"换空间一次性降维"。

这种解耦还带来一个重要的工程红利:"what to generate" 与 "how to render" 完全分离。latent diffusion 决定"内容"(构图、对象、姿态),frozen decoder 决定"渲染"(纹理、笔触、风格细节)。想换风格?不重训 U-Net,只 fine-tune decoder 就行——这就是 SDXL refiner、各种社区 VAE finetune(mse-ft、ema-ft)能存在的原理。反过来,想换内容理解?保留 decoder,只重训 U-Net(DreamBooth、LoRA 走的路径)。两条工程路径互不干扰,是 latent diffusion 生态繁荣的底层原因。

🟡 教材外补充:源 CMU L11 没展开 latent diffusion 细节,但 Alex 列举的 Luma R3、Midjourney、Flux 全部建立在 Rombach 2022 这套两阶段框架上。SD1/SD2 用 KL-VAE(4 channel);SDXL 升级到 4 channel 但更大空间分辨率 + refiner 二级 U-Net;SD3 改用 16 channel VAE 并把 backbone 换成 MMDiT (multimodal diffusion transformer),丢弃 U-Net;FLUX.1 用 16 channel + rectified flow 替代 DDPM;Luma R3、Sora 都把 latent 扩到 3D (时间维)。共同点是没有人回到像素空间——latent diffusion 已经是 2026 年视觉生成事实上的标配。
x 512×512×3 E VAE enc frozen z₀ 64×64×4 U-Net 去噪 z_T → z_{T-1} → … → z_0 cross-attn(text c) T steps D VAE dec frozen x̂ = D(z₀) text c (CLIP/T5)

下面按顺序点四个按钮,看 latent diffusion 一次完整的 训练 / 推理 流程。不变量:所有运算都在 $64 \times 64 \times 4$ 的 latent 上进行,pixel space 只在最起点和最终点出现两次。

z₀ = E(x₀) ∈ ℝ^{64×64×4}  // 像素 → latent,一次性,frozen
z_t = √ᾱₜ · z₀ + √(1−ᾱₜ) · ε,   ε ∼ 𝒩(0, I)  // 在 latent 加噪
z_{t−1} ← μ_θ(z_t, t, c) + σ_t · ξ,   for t = T, T−1, …, 1  // U-Net 反向 T 步
x̂ = D(z₀) ∈ ℝ^{512×512×3}  // latent → 像素,一次性,frozen

手算压缩比与算力比

Pixel 维度 $= 512 \cdot 512 \cdot 3 = 786{,}432$。Latent 维度 $= 64 \cdot 64 \cdot 4 = 16{,}384$。压缩比 $= 786{,}432 / 16{,}384 = 48$。

注意力空间 token 数:pixel 端 $N_p = 512 \cdot 512 = 262{,}144$;latent 端 $N_z = 64 \cdot 64 = 4{,}096$。自注意力复杂度 $\propto N^2$,故比值 $= (262{,}144 / 4{,}096)^2 = 64^2 = 4{,}096$。算上卷积部分被稀释,实际端到端加速实测约 $50\times$,显存下降约 $10\times$(Rombach 2022 Table 8)。

训练 cost 对比(512×512、A100 单卡):pixel DDPM ≈ 150 GPU-day;SD-1.4 ≈ 6 GPU-day 在 latent 上即可达到同等 FID。差 $25\times$。

再算一笔显存账:U-Net 前向时单层 activation $\propto N \cdot c$,pixel 端单 channel 单层 $= 262{,}144 \cdot 4\text{ bytes} = 1\text{ MB}$,latent 端 $= 16{,}384 \cdot 4 = 64\text{ KB}$。一个 32 层 U-Net、256 channel、batch=4,pixel 端 activation 总量 $\approx 32 \cdot 256 \cdot 1\text{MB} \cdot 4 = 32\text{ GB}$(爆显存);latent 端 $\approx 2\text{ GB}$(轻松塞进 8GB 卡)。这就是为什么 SD 能在 RTX 3060 上跑而 pixel DDPM 不能。

常见误区

  1. "Latent diffusion 是端到端训 VAE + U-Net"。反例:Stable Diffusion 1.4 的 config.yaml 里 first_stage_config.params.ckpt_path 指向一个独立预训练的 KL-VAE(kl-f8.ckpt),训 U-Net 时 first_stage_model.eval() 全程开着、梯度不回传。如果你尝试解冻 VAE 一起训,会发现 latent 分布漂移、训练初期重建质量明显崩坏,FID 在前 10k 步内反而上升——因为 diffusion 的梯度信号是对"已成形的语义压缩空间"去噪,强行修改这个空间等于让 U-Net 追着一个移动靶打。SD3 后续尝试过端到端联合微调,也只是在 U-Net 训练后期解冻 VAE 做有限步数的微调,全程冻结仍是默认方案。
  2. "latent 越小越好,反正算得快"。反例:把 $f$ 从 8 调到 16(latent $32 \times 32 \times 4$),FID 从 8.3 涨到 17.6(Rombach Table 6);继续到 $f=32$,FID 飙到 27。压得太狠丢失重建细节,diffusion 再准也救不回——decoder 是上限,VAE 重建 FID 是 latent diffusion FID 的不可突破下界。这告诉我们:"perceptual compression 的甜区"必须实测扫出来,不能拍脑袋拉大压缩比。反过来 $f=4$ 重建质量虽好,但 latent 维度回到 $128 \times 128 \times 3 \approx 49\text{K}$ 维,diffusion 算力上反而失去了主要优势。
  3. "VAE-latent ≈ 高斯,所以可以直接采 $z \sim \mathcal{N}(0, I)$ 拿去 decode"。反例:实测 latent 的 channel-wise std ≈ $\{4.17, 3.51, 4.06, 3.36\}$(SD-1.4 量化结果),不是单位方差;Rombach 在配置里专门加了 scale_factor=0.18215(约等于 $1/\text{std}$)把 latent 重标定到接近单位方差,diffusion 才稳。直接拿 $\mathcal{N}(0, I)$ 采样会 decode 出一片灰雾(亮度均值正确但纹理全乱)。Stable Diffusion 推理代码里 z = z / 0.18215 这行就是反归一化,少了它整张图爆白。这个常数不是从理论推出来的,是 Rombach 在训练前先跑一批数据统计出 latent 标准差再取倒数得到的经验值——换数据集或换 VAE checkpoint 就要重新统计。

相关概念

Latent diffusion 是一个连接性极强的概念:它向上承接 VAE 的压缩理论,向下接管所有现代采样技巧,横向延伸到视频生成。理解了它,下面几个概念基本就连成网了。

  • Video Diffusion — 把同一套两阶段框架扩到时序,latent 从 4D 变成 5D(含时间维),Luma R3、Sora、Veo 走的都是这条路。
  • Classifier-Free Guidance — Stable Diffusion 推理的标配条件强化技术,在 latent 空间执行 $\epsilon_{\text{guided}} = \epsilon_\text{uncond} + w(\epsilon_\text{cond} - \epsilon_\text{uncond})$,是文图对齐质量的关键调控旋钮。
  • DDIM — Stable Diffusion 默认采样器,在 latent 上跑同样适用,把推理步数从 1000 降到 50 是商用落地的前提。
  • Probability Flow ODE — 把 latent diffusion 改写成 ODE 后能套 DPM-Solver 这类高阶求解器,进一步把步数压到 20 步以下。

来源

  1. cmu-10799-diffusion-flow · L11 · Guest Lecture Linqi (Alex) Zhou (Luma AI) — diffusion/flow 主宰视觉生成 · (00:44)
  2. cmu-10799-diffusion-flow · L11 · 商用 T2I/T2V 模型(Midjourney、Flux、Luma R3)背后都是 diffusion · (01:36)
  3. cmu-10799-diffusion-flow · L11 · Simple L2 loss 与 inference 慢的根因 · (04:13)
查看 83 条课程原文出处
CourseLectureTimeExcerpt
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 00: 03 Cool. Cool. Yeah. Uh and thanks again Cool. Cool. Yeah. Uh and thanks again for the invite. Um I'm very excited to for the invite. Um I'm very excited to for the invite. Um I'm very excited to see you…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 00: 44 done without my fantastic colleagues at luma. Yeah. luma. Yeah. luma. Yeah. So um in the past five years or so uh So um in the past five years or so uh So um in the past five years or so uh diffusion …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 01: 36 Yeah, there's some jitter. But um yeah, Yeah, there's some jitter. But um yeah, we we show some samples from our latest we we show some samples from our latest we we show some samples from our latest …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 02: 26 uh so to visualize this process you know uh so to visualize this process you know on a on a simple 2D graph um we show on a on a simple 2D graph um we show on a on a simple 2D graph um we show that he…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 03: 21 And these trajectories can be seen as And these trajectories can be seen as these uh blue lines uh here. And the these uh blue lines uh here. And the these uh blue lines uh here. And the ground truth …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 04: 13 and uh ultimately the loss comes down to a simple L2 loss uh which is extremely a simple L2 loss uh which is extremely a simple L2 loss uh which is extremely stable and very scalable to large number s…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 05: 10 naturally it suffers from first uh the OD simulation error uh and slow OD simulation error uh and slow OD simulation error uh and slow inference inference inference so um so um so um so as a result uh…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 06: 03 scaling and this can be best uh represented by the green line here where represented by the green line here where represented by the green line here where where you know when you have very few where y…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 07: 00 problem uh with these types of models is that uh first it it can often suffer that uh first it it can often suffer that uh first it it can often suffer from training instability. For example, from tra…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 07: 55 these different tasks we need to do step step distillation Um and this usually step distillation Um and this usually step distillation Um and this usually comes after the RL or preference or comes aft…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 08: 43 today I'll be cover two works that we did at luma uh which is inductive moment did at luma uh which is inductive moment did at luma uh which is inductive moment matching uh and terminal velocity match…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 09: 36 yeah let's first look at uh how we can achieve efficient inference by design. So to think about efficient inference uh So to think about efficient inference uh we we first need to look at what makes w…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 10: 35 your neuronet network taking in XT&T. And the result the the result of one And the result the the result of one And the result the the result of one step DDIM uh at time S can be best step DDIM uh at …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 11: 34 relationship. Um so we we I I I like to term ter term Um so we we I I I like to term ter term Um so we we I I I like to term ter term this the capacity problem uh of DDIM. this the capacity problem uh…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 12: 32 So so that comes with the power that's uh that's from the um the neuronet uh that's from the um the neuronet uh that's from the um the neuronet network that that's being injected with network that tha…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 13: 34 >> Okay. Do you guys have questions regarding the first part? regarding the first part? regarding the first part? >> No >> No >> No question. question. question. >> Okay. Okay. Yeah. Like I I I told A…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 14: 12 model on S then we can like s like hope model on S then we can like s like hope that it learns the OD better. that it learns the OD better. that it learns the OD better. >> So what about let's say uh …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 15: 03 neural network you are still trying to fit the tangent velocity at every single fit the tangent velocity at every single fit the tangent velocity at every single point. >> I hope that answers the ques…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 16: 00 few step uh network uh directly from from scratch. Okay. So the first method uh inductive Okay. So the first method uh inductive mode matching. So for for this uh for mode matching. So for for this uh…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 17: 05 and we call the two intermediate distributions uh QS and QT where where distributions uh QS and QT where where distributions uh QS and QT where where QS is closer to our data distribution QS is closer…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 18: 00 to to learn this mapping uh that transforms QT to QS uh a naive approach transforms QT to QS uh a naive approach transforms QT to QS uh a naive approach right to learn this mapping uh is is to right t…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 19: 08 is it is kind of kind of like a GAN but is it is kind of kind of like a GAN but uh the optimal discriminator is already uh the optimal discriminator is already uh the optimal discriminator is already …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 20: 12 And um a second component uh for IM is the concept of inductive learning. Uh so the concept of inductive learning. Uh so the concept of inductive learning. Uh so consider the same learning scheme wher…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 21: 01 that is the case here when QT is very far away from from QS. Um so naively far away from from QS. Um so naively far away from from QS. Um so naively matching these two distributions with matching thes…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 21: 58 QQS. um and and and learning the model um and and and learning the model um and and and learning the model learning the model distribution uh learning the model distribution uh learning the model dist…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 22: 54 So what is the base case here? The base case is if you take QS. case is if you take QS. case is if you take QS. Oh uh yeah and here we need to yeah Oh uh yeah and here we need to yeah Oh uh yeah and h…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 23: 55 true regardless of your theta and even true regardless of your theta and even when your theta is untrained um that the when your theta is untrained um that the when your theta is untrained um that the…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 24: 53 is going to be the base case itself. So this is going to be identically equal to this is going to be identically equal to this is going to be identically equal to QS. QS. QS. So what you're matching a…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 25: 51 >> question. Yeah, >> about the choice of MMD. I mean uh we >> about the choice of MMD. I mean uh we >> about the choice of MMD. I mean uh we can formalize the the flow matching as can formalize the t…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 26: 40 some iterative scheme uh to to actually optimize uh that objective. So yeah, so optimize uh that objective. So yeah, so optimize uh that objective. So yeah, so that's part of the reason that we don't …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 27: 31 RBF and and uh Llas kernel both work in RBF and and uh Llas kernel both work in our case. our case. our case. >> Yeah. Yeah. Thank you. >> Yeah. Yeah. Thank you. >> Yeah. Yeah. Thank you. >> Yeah. Yea…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 28: 20 target to to train the new the new time target to to train the new the new time step um the the new model distribution step um the the new model distribution step um the the new model distribution and…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 29: 11 and improved consensity training uh in discrete time. Uh and this is done on discrete time. Uh and this is done on discrete time. Uh and this is done on imageet. So consensity training imageet. So con…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 30: 14 transformer size. Uh so so so the more compute you throw at it, it gets uh compute you throw at it, it gets uh compute you throw at it, it gets uh better performance in general better performance in g…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 31: 09 to consistency training as a special case if our MMD objective uses only one case if our MMD objective uses only one case if our MMD objective uses only one particle. Uh and instead of using LLAS part…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 32: 04 and G GMMN can also be seen as a special and G GMMN can also be seen as a special case of Imm. Uh case of Imm. Uh case of Imm. Uh you can easily see that when if you set you can easily see that when i…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 32: 53 essentially in this paper the drift field for their uh for their model is field for their uh for their model is field for their uh for their model is also parameterized via uh via MMD also parameteriz…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 33: 53 per GPU can can usually be as low as per GPU can can usually be as low as one. uh and in this case you know you one. uh and in this case you know you one. uh and in this case you know you need to gath…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 34: 45 distinguish between nearby R&T then it distinguish between nearby R&T then it it's generally uh harder to scale. it's generally uh harder to scale. it's generally uh harder to scale. So these are so t…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 35: 36 so kernels is is a very uh standard um concept I guess in machine learning um concept I guess in machine learning um concept I guess in machine learning um it's it it's basically a a similarity it's i…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 36: 34 to to some um vector where each entry is like a uh a power of this real number like a uh a power of this real number like a uh a power of this real number and that is kind of a augmented feature and t…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 37: 28 the and the and the keys right and this is sort of like basically we project the is sort of like basically we project the is sort of like basically we project the query and the keys to some feature sp…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 38: 15 and also maybe like if you guys have tried like empirically like how do would tried like empirically like how do would tried like empirically like how do would you compare the three types of models? y…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 39: 20 point XR say. Um so what what we found point XR say. Um so what what we found is that uh because of the correlation is that uh because of the correlation is that uh because of the correlation through …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 40: 19 particle for training uh for for for particle for training uh for for for training the model means that they are training the model means that they are training the model means that they are ignoring …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 41: 16 >> uh, yeah, but but for the next work, I guess it'll be much closer to flow maps. guess it'll be much closer to flow maps. guess it'll be much closer to flow maps. >> All right. Cool. Cool. >> All ri…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 42: 10 curved path in space which I'm sure all of you know um and on the right uh we of you know um and on the right uh we of you know um and on the right uh we have uh have uh have uh a a model train trend …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 43: 11 prior x1. So the tangent along this OD trajectory So the tangent along this OD trajectory So the tangent along this OD trajectory is the marginal velocity is the marginal velocity is the marginal velo…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 44: 15 displacement f theta uh which we can define um you know as uh uh which we can define um you know as uh uh which we can define um you know as uh s minus t * capital f where where s minus t * capital f …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 45: 11 map that directly takes xt to x0 map that directly takes xt to x0 so naively uh what we want to learn oh so naively uh what we want to learn oh so naively uh what we want to learn oh how How we can le…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 46: 00 actually want to look at is this quantity um which which characterizes quantity um which which characterizes quantity um which which characterizes the the terminal velocity uh the the the the terminal…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 47: 03 if it happens to match the marginal velocity at this point then you have velocity at this point then you have velocity at this point then you have automatically learned this this automatically learned…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 48: 07 and this is best shown uh in in this figure uh in the previous slides. Um so figure uh in the previous slides. Um so figure uh in the previous slides. Um so this is the case when when uh the bound thi…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 49: 06 these quantities exist but we don't know that during pre pre-training time so how that during pre pre-training time so how that during pre pre-training time so how do we so how do we kind of have acce…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 50: 02 your own network u approximates the the your own network u approximates the the ground truth velocity at at every single ground truth velocity at at every single ground truth velocity at at every sing…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 50: 56 that uh TVM loss upper bounds the two that uh TVM loss upper bounds the two wound distance up to some constant and wound distance up to some constant and wound distance up to some constant and this is…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 51: 54 actually not lies continuous because of the dot product self attention and layer the dot product self attention and layer the dot product self attention and layer norms and there's actually a lot of n…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 52: 51 we you know add add RMS norm as uh we you know add add RMS norm as uh without any parameters uh to the time without any parameters uh to the time without any parameters uh to the time step modulation …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 53: 53 and you can do some quick mental math uh to just take derivative with respect to to just take derivative with respect to to just take derivative with respect to s on both sides, right? And s on both s…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 54: 49 have any stop gradient um which means have any stop gradient um which means that we ideally want to back propagate that we ideally want to back propagate that we ideally want to back propagate through…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 55: 39 So this is actually exactly opposite of meanflow which takes partial on both xt meanflow which takes partial on both xt meanflow which takes partial on both xt and t. and t. and t. Yeah. Uh so I will …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 56: 38 very stable. And if we track also the norm of the And if we track also the norm of the And if we track also the norm of the marginal velocity uh for both mean flow marginal velocity uh for both mean f…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 57: 32 network predicted velocity and uh and um network predicted velocity and uh and um conditional velocity and there are conditional velocity and there are conditional velocity and there are additional va…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 58: 34 >> and yeah um experimentally we achieve still there one step result on imaget still there one step result on imaget still there one step result on imaget and it can also surpass uh dn and xit and it …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 59: 27 scale and this model is is trained with scale and this model is is trained with pure TVM loss uh closely following what pure TVM loss uh closely following what pure TVM loss uh closely following what …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 60: 16 for uh FSTP if you naively call JVP on for uh FSTP if you naively call JVP on the entire FSTP sharded model the entire FSTP sharded model the entire FSTP sharded model uh then FSTP will usually error …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 61: 07 first challenge with uh FSTP with JVP first challenge with uh FSTP with JVP and the second challenge was is uh is and the second challenge was is uh is and the second challenge was is uh is writing JV…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 61: 58 uh I will take some questions. Uh but before that I will I have also have I before that I will I have also have I before that I will I have also have I was also given a task of of recruiting. was also…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 62: 51 reach out to me or to to our our team and uh also the inventors of DDIM and and uh also the inventors of DDIM and and uh also the inventors of DDIM and Nerf both both work here. So, the mentor Nerf bo…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 63: 46 >> may I ask question so I might be wrong but it seems that when you do the but it seems that when you do the but it seems that when you do the terminal velocity matching at a specific terminal veloci…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 64: 27 time team? Oh uh do you mean the integration integration integration here? here? here? >> Yeah. Over a different time. >> Yeah. Over a different time. >> Yeah. Over a different time. >> Oh yeah. Yeah.…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 65: 17 the the integration only comes in um if the the integration only comes in um if you want to measure its relationship you want to measure its relationship you want to measure its relationship with resp…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 66: 14 any sweet spot for the NFP like with four NFP you are getting good quality four NFP you are getting good quality four NFP you are getting good quality but if you kind of increase it to 50, but if you …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 67: 03 scratch like it's just a distillation approach how is it compared to EVM approach how is it compared to EVM approach how is it compared to EVM >> uh sorry for do you mean comp compared >> uh sorry for…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 67: 56 your own network then then that's basically a dissolution technique. Yeah. basically a dissolution technique. Yeah. basically a dissolution technique. Yeah. >> Thank you. >> Thank you. >> Thank you. >…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 68: 52 step sampling and if you spend more more more compute on on more intermediate more compute on on more intermediate more compute on on more intermediate step it doesn't give you a huge gain um step it …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 69: 54 inspired me uh to write this paper was a inspired me uh to write this paper was a work called um uh physics informed work called um uh physics informed work called um uh physics informed dissolation f…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 70: 48 know. >> Is there any computational overhead >> Is there any computational overhead >> Is there any computational overhead since we have a much more complex final since we have a much more complex fin…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 71: 50 objectives at the same at the same time. objectives at the same at the same time. Um uh so we have observed that if you Um uh so we have observed that if you Um uh so we have observed that if you opti…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 72: 39 terms of performance or in terms of like terms of performance or in terms of like training stabilities like like yeah training stabilities like like yeah training stabilities like like yeah basically …
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 73: 34 models in general still has some advantage in terms of training training advantage in terms of training training advantage in terms of training training uh uh training simplicity I guess. Um uh uh tra…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 74: 32 >> no. Uh but that's a great uh I think to to study uh because for we only used the to study uh because for we only used the to study uh because for we only used the DDIM uh uh the the the DDM form of…
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 75: 28 out. >> Yeah, please reach out. Um yeah and Alex >> Yeah, please reach out. Um yeah and Alex >> Yeah, please reach out. Um yeah and Alex can if you can share with me with the can if you can share with…