概念

SDE Formulation

— 把 DDPM / SMLD / Langevin 同时塞进 $dx = f\,dt + g\,dW$ 这一条连续时间方程。

很多人第一次听说 "SDE formulation" 会以为这是 DDPM 之外的另一种扩散模型——"哦,又有一种新方法叫 SDE"。这是最大的误读。SDE formulation 不是一个新模型,它是 DDPM、SMLD 和 Langevin 动力学的连续时间极限,把这些原本看起来各自独立的离散方法统一进同一条随机微分方程 $dx = f(x, t)\,dt + g(t)\,dW$ 之下。你在 SDE 视角下不会得到一种"新的扩散模型",但你会突然看清:DDPM 不过是 VP-SDE 的 Euler-Maruyama 离散;SMLD 不过是 VE-SDE 的离散;Langevin 动力学不过是反向 SDE 在去掉漂移项后的特例。这一统一是 Yang Song 2021 那篇 ICLR best paper 的核心贡献。

Vizuara 的 Dr. Raj 在 Lecture 8 把这套统一称作"把所有东西串起来的框架",他在开场就直接把这层身份说出来:

"We have so far covered VAEs then DDPMs ... then the energy-based approach which includes score function and noise conditioned score network. Today we are going to look at a framework which unifies everything together." —— Vizuara《Principles of Diffusion Models》L8 @00:05

注意 Dr. Raj 强调的"unifies everything"——VAE / DDPM / score-based 这三条主线都在 SDE 之下归一。他随后给出本节最重要的物理直觉:

"By diffusion we understand that something is spreading in space ... if you have a background in physics or mechanics, particles, fluid mechanics, you'll find this lecture very interesting." —— Vizuara L8 @02:06

他用一杯水里滴一滴红色染料作为整节课的中央比喻:染料起初是球形,落进水里,慢慢扩散,最终整杯水变红、原来的形状完全被破坏。然后他抛出本节真正要回答的问题——能不能反过来?给你这一杯均匀的红水,能不能恢复出原来那滴染料的形状?宏观上不行(时间不可逆),但微观上的布朗运动是时间可逆的,这一条物理事实就是整个扩散模型生成的根基。Dr. Raj 把这件事讲得很重:

"It turns out that even though macroscopic motions are not time reversible ... at the microscopic level, if you look at these individual tiny particles, ... these motions are exactly reversible. ... this is the only theme which is used in the entire diffusion literature in all the different approaches." —— Vizuara L8 @07:30

"the only theme"——所有扩散文献都用这一条原理。Anderson 1982 把这条物理事实写成了数学定理:给定前向 SDE $dx = f\,dt + g\,dW$,存在一条对应的反向 SDE,其漂移项里多出一个 $-g^2 \nabla_x \log p_t(x)$,即 score function。这就是"网络要学的东西"——score。

CMU 10-799 那门课在 SDE 视角下走得更形式化,直接从 ODE 出发先讲一遍连续时间的流向,再把布朗噪声叠加进去得到 SDE,最后把 Anderson 反向 SDE 当成必要的桥梁推出 PF-ODE。两门课讲法不同,但关键身份是一致的:DDPM / SMLD / Langevin 在 SDE 下变成同一件事的三种离散化。

在训练里这件事对应:把 DDPM 的 $T = 1000$ 步极限拉到 $T \to \infty$、$\beta_t \to 0$,得到连续时间 VP-SDE;网络的目标依然是预测 score(在 $\epsilon$-prediction 参数化下,score 和 $\epsilon$ 只差一个 $-\sqrt{1-\bar\alpha_t}$ 系数)。SDE 视角不改训练,但它解放推理——你可以用任何高阶 SDE/ODE solver、任何步长(甚至自适应),不再被原来 DDPM 的离散链锁死。

1. 前向 SDE:一行方程容纳所有扩散

SDE formulation 的全部前向过程写成一条方程:

$$dx = f(x, t)\,dt + g(t)\,dW$$

其中 $f(x, t)$ 称作漂移项(drift),$g(t)$ 称作扩散系数(diffusion coefficient),$W$ 是标准布朗运动(Wiener process)。$dt \to 0$ 时这条方程定义了一条连续随机轨迹。当 $f, g$ 取不同形式,就得到不同的扩散模型族。Song et al. 2021 论文里给出两个最重要的特例:

1.1 VP-SDE (Variance Preserving):DDPM 的连续极限

$$f(x, t) = -\tfrac{1}{2}\beta(t)\,x, \qquad g(t) = \sqrt{\beta(t)}$$

展开来看:$dx = -\tfrac{1}{2}\beta(t)\,x\,dt + \sqrt{\beta(t)}\,dW$。

Why "variance preserving":直接验证一下。如果 $x_0$ 单位方差,$\text{Var}(x_t)$ 在这条 SDE 下的演化由伊藤公式给出 $d\,\text{Var} = (-\beta(t)\,\text{Var} + \beta(t))\,dt = \beta(t)(1 - \text{Var})\,dt$。这是个一阶线性 ODE,平衡点是 $\text{Var} = 1$——也就是说方差被驱动到 1 并保持。$t \to T$ 时 $x_T \to \mathcal{N}(0, I)$。这是 DDPM 离散版"$\sqrt{1-\beta_t}$ 系数让总方差守恒"的连续时间镜像。Vizuara 的染料比喻在这里对应:染料的"信号量"被持续磨平,但水的"总体浓度尺度"始终是单位 1。

1.2 VE-SDE (Variance Exploding):SMLD 的连续极限

$$f(x, t) = 0, \qquad g(t) = \sqrt{\frac{d\sigma^2(t)}{dt}}$$

展开来看:$dx = \sqrt{d\sigma^2/dt}\,dW$,没有漂移,只有不断累积的噪声。

Why "variance exploding":直接积分得到 $\text{Var}(x_t) = \text{Var}(x_0) + \sigma^2(t) - \sigma^2(0)$,方差随时间无限增长。Song & Ermon 2019/2020 的 NCSN / SMLD 用一组几何序列 $\sigma_1 < \sigma_2 < \cdots < \sigma_L$($\sigma_L$ 可大到 50 甚至 80)训一个 noise-conditioned score network。连续极限下就是 VE-SDE。$x_T$ 在 VE-SDE 下不收敛到 $\mathcal{N}(0, I)$,而是变成 $\mathcal{N}(x_0, \sigma_T^2 I)$——一张极其 noisy 的图,几乎跟原图无关。

VP-SDE(DDPM 系)

$f = -\tfrac{1}{2}\beta(t)\,x$,$g = \sqrt{\beta(t)}$。
方差驱动到 1。
$x_T \approx \mathcal{N}(0, I)$。
$\beta(t)$ 是 schedule(DDPM linear / cosine 的连续版本)。

VE-SDE(SMLD 系)

$f = 0$,$g = \sqrt{d\sigma^2/dt}$。
方差无限膨胀。
$x_T \approx \mathcal{N}(x_0, \sigma_T^2 I)$,$\sigma_T$ 很大。
$\sigma(t)$ 通常取 $\sigma_{\min}(\sigma_{\max}/\sigma_{\min})^t$,几何递增。

论文还给出第三个sub-VP SDE,它在 VP 基础上微调 $g(t)$,让中段方差略小,likelihood 表现更好;工程上一般直接用 VP 或 VE,sub-VP 的提升不大。

VP-SDE 与 VE-SDE:方差随 t 的不同命运 t Var(x_t) Var=1 VP-SDE:方差被拉到 1(信号-噪声此消彼长) VE-SDE:方差无限膨胀(只加噪声不缩信号) t=0 t=T x_0

2. 反向 SDE:Anderson 1982 的魔法

有了前向 SDE 还不够——生成需要的是反向过程:从噪声出发回到数据。Brian Anderson 在 1982 年的论文《Reverse-time diffusion equation models》证明了如下事实:给定任意 Itô SDE $dx = f(x, t)\,dt + g(t)\,dW$,它在时间反演下对应一条反向 SDE:

$$dx = \left[\,f(x, t) - g(t)^2 \nabla_x \log p_t(x)\,\right] dt + g(t)\,d\bar W$$

其中 $\bar W$ 是反向布朗运动,$\nabla_x \log p_t(x)$ 是时间 $t$ 处边际分布 $p_t(x)$ 的score function。

Why this is the entire trick:反向 SDE 里除了 score 之外的所有项都是已知的——$f, g$ 是你自己设计的,$\bar W$ 是随机数。唯一未知的就是 $\nabla_x \log p_t(x)$。score 一旦学到,反向 SDE 就完全可执行;生成等价于"在 $t = T$ 起点采 $x_T \sim$ 先验(VP 是 $\mathcal{N}(0, I)$,VE 是大方差高斯),然后离散 Anderson SDE 一路积分到 $t = 0$"。这把"生成"这个抽象任务规约到了"学一个向量场"这件具体的事。Dr. Raj 那句"this is the only theme"在这里数学化了——所有扩散模型的生成都来自 Anderson 这一公式。

score 是什么:$\nabla_x \log p_t(x)$ 是密度函数对位置的对数梯度,它指向密度更高的方向。生成时反向 SDE 顺着 score 走,相当于"沿着数据流形的引力上行",每一步还叠加一点点反向噪声 $g(t)\,d\bar W$,让样本不收敛到 mode 而保持多样性。

3. 从离散到连续:DDPM 即 Euler-Maruyama 离散

SDE 视角真正强大的地方,是它把 DDPM 这个看起来"很离散、链式、$T = 1000$ 步"的算法识别为 VP-SDE 用最简单的数值方法离散后的版本:

$$x_{t+\Delta t} = x_t + f(x_t, t)\,\Delta t + g(t)\,\sqrt{\Delta t}\,z, \qquad z \sim \mathcal{N}(0, I)$$

这就是Euler-Maruyama 方法——把 $dt$ 替换成有限步长 $\Delta t = 1/T$。代入 VP 的 $f = -\tfrac{1}{2}\beta(t)x$, $g = \sqrt{\beta(t)}$,离散后正好等价于 DDPM 的 $q(x_{t+1}|x_t)$ 一步加噪。

Why this matters:Euler-Maruyama 是最低阶的 SDE 离散方法(一阶强收敛 $O(\sqrt{\Delta t})$)。DDPM 用 $T = 1000$ 步就是因为低阶方法误差大,必须把步长压到 $10^{-3}$ 才能保证精度。一旦你认出 DDPM 是 SDE 的一阶离散,立刻就能换更好的离散方法:

  • Heun / Runge-Kutta(高阶):每步多算一次 score,但精度提升让总步数从 1000 降到 30–50(EDM Karras 2022 就走这条线)。
  • DPM-Solver(Lu et al. 2022):用 SDE 的半解析解 + 高阶 Taylor 展开,20 步即可逼近 1000 步的质量。
  • 自适应步长:在 $t$ 接近 0(细节关键区)密一些,在 $t$ 接近 $T$(纯噪声区)稀一些。

这些采样器都不需要重新训练——只需要把已经学到的 score 网络换个数值积分器跑一遍。这是 SDE 视角第一个"免费午餐"。

4. 训练目标:score matching 的连续版本

训练目标在 SDE 视角下统一为 continuous-time score matching:

$$\mathcal{L}(\theta) = \mathbb{E}_{t \sim \mathcal{U}[0, T]}\left[\lambda(t)\,\mathbb{E}_{x_0, x_t}\big[\,\|s_\theta(x_t, t) - \nabla_{x_t} \log p_t(x_t | x_0)\|^2\,\big]\right]$$

其中 $s_\theta$ 是网络("score network"),$\lambda(t)$ 是 loss 权重(取不同 $\lambda$ 即对应 DDPM 的 simple loss、likelihood weighting、EDM 的精心调权)。$\nabla \log p_t(x_t | x_0)$ 有闭式——因为给定 $x_0$,$x_t$ 服从已知的高斯分布(VP / VE 都给了闭式过渡核),所以 score 直接可计算,无需估计 $p_t(x_t)$ 本身。这正是 Vincent 2011 denoising score matching 的连续时间推广。

在 VP-SDE 下,$x_t = \sqrt{\bar\alpha(t)}\,x_0 + \sqrt{1 - \bar\alpha(t)}\,\epsilon$,于是 $\nabla \log p_t(x_t | x_0) = -\epsilon / \sqrt{1 - \bar\alpha(t)}$。这就直接和 DDPM 的 $\epsilon$-prediction 对应了——score 和 $\epsilon$ 只差一个时间相关的标量系数:

$$s_\theta(x_t, t) = -\frac{\epsilon_\theta(x_t, t)}{\sqrt{1 - \bar\alpha(t)}}$$

Why this is "everything unifies":DDPM 的 simple loss、SMLD 的 NCSN loss、score matching 的 denoising 形式,在 SDE 视角下都是同一个目标——learning the score——只是参数化($\epsilon$ / $s$ / $x_0$)和权重 $\lambda(t)$ 不同。这就是 Dr. Raj 所说"这一框架把所有东西统一起来"的数学落地。

🟡 教材外补充:Karras et al. 2022 的 EDM 论文进一步推广这套统一——它指出 parameterization、weighting、preconditioning 是三个独立的设计自由度,过去的 DDPM / SMLD / score matching 只是在这三个自由度的不同角点。EDM 的标准化做法($\sigma$-parameterization + $w(\sigma)$ weighting + skip connection preconditioner)几乎在所有图像生成 benchmark 上比原 DDPM / SMLD 高 1–2 个 FID 点,是 SDE 视角"统一后再重新选最优"的工程胜利。

5. PF-ODE:deterministic 镜像

Song 2021 的另一个核心结果:反向 SDE 有一个确定性的双胞胎,叫 probability flow ODE(PF-ODE):

$$\frac{dx}{dt} = f(x, t) - \tfrac{1}{2}\,g(t)^2\,\nabla_x \log p_t(x)$$

注意系数变成了 $\tfrac{1}{2}\,g^2$(反向 SDE 是 $g^2$),并且去掉了 $g\,d\bar W$ 这一项——整条 ODE 不再有随机性。神奇之处在于:从同一个 $x_T$ 出发,PF-ODE 与反向 SDE 在边际分布意义下产生相同的 $p_0$,但 PF-ODE 是确定性的,意味着同一个噪声种子永远生成同一张图,可以用任何 ODE solver(RK4、DPM-Solver 等)求解。

Why two flavors:SDE 反向因为还有噪声项,生成多样性更高、模式覆盖更好;PF-ODE 确定但有时模式坍缩到训练集 mode,多样性略差。tradeoff:步数越多(步长越小)越接近真实 SDE / ODE 解;步数少时 SDE 的随机噪声反而能"踢出"误差累积,所以低 NFE(function evaluations)下 SDE 经常优于 ODE。DDIM 就是 PF-ODE 在 VP-SDE 上的一阶离散(Song 2020 的 DDIM 论文先有,2021 SDE 论文之后才完整理解)。

6. 与已有方法的身份对照

把所有方法塞进 SDE 视角下的"门牌":

  • DDPM (Ho 2020) = VP-SDE 的 Euler-Maruyama 离散,$T = 1000$,$\epsilon$-parameterization。
  • SMLD / NCSN (Song & Ermon 2019/2020) = VE-SDE 的离散,几何噪声序列,score-parameterization。
  • Langevin dynamics = 反向 SDE 在 $f = 0$、固定 $t$(不让 score 随时间变)下的特例:$dx = \tfrac{1}{2}\eta\nabla \log p(x)\,dt + \sqrt{\eta}\,dW$。可以理解为"在每个噪声等级上多走几步 Langevin 来 polish"——NCSN 的 annealed Langevin 采样就是这件事。
  • DDIM (Song 2020) = PF-ODE 在 VP-SDE 上的一阶离散,deterministic、可少步采样。
  • EDM (Karras 2022) = SDE 视角下重新选择 parameterization / weighting / preconditioning 的最优组合。
  • Flow Matching / Rectified Flow = 在 SDE 中令 $g = 0$(退化为 ODE)并直接学 vector field,绕开 score 学习。

这张表是理解 2020–2024 年扩散文献谱系的关键索引。每读一篇新论文,先问:它在 $(f, g, \lambda(t), \text{solver})$ 这四个轴上占什么位置?答案出来,定位完成。

下面四个按钮把 SDE 视角的核心四步走依次展开。不变量:每一步都同时对 VP 和 VE 成立——只是 $f, g$ 的具体形式不同,整套推理结构是同一个。这正是 SDE formulation 的统一力所在。

手算 VP-SDE 一步 Euler-Maruyama

取 VP-SDE $f = -\tfrac{1}{2}\beta(t)\,x$,$g = \sqrt{\beta(t)}$,$\beta(t) = 0.1$(取常数简化),步长 $\Delta t = 0.01$。设当前 $x_t = 0.8$,抽到 $z \sim \mathcal{N}(0, 1)$ 这次拿到 $z = 0.3$:

$$x_{t+\Delta t} = x_t + f\,\Delta t + g\,\sqrt{\Delta t}\,z = 0.8 + (-\tfrac{1}{2} \cdot 0.1 \cdot 0.8) \cdot 0.01 + \sqrt{0.1} \cdot \sqrt{0.01} \cdot 0.3$$

$$= 0.8 + (-0.04) \cdot 0.01 + 0.316 \cdot 0.1 \cdot 0.3 = 0.8 - 0.0004 + 0.00948 = 0.80908$$

注意三件事:(1) 漂移项 $-0.0004$ 把 $x$ 微微往零拉("收缩信号");(2) 扩散项 $+0.00948$ 把它推开一点("撒胡椒");(3) 漂移项 $\propto \Delta t$,扩散项 $\propto \sqrt{\Delta t}$——这就是 SDE 区别于 ODE 的本质,noise 用 $\sqrt{\Delta t}$ 而不是 $\Delta t$ 缩放,因为布朗运动的方差线性增长(变化幅度是其平方根)。这条尺度差异是 Euler-Maruyama 与 Euler 的唯一区别,也是为什么 SDE 离散收敛阶比 ODE 低($O(\sqrt{\Delta t})$ vs $O(\Delta t)$)。

把 $T = 1$、$\Delta t = 0.001$(即 $T_{\text{discrete}} = 1000$ 步)就完全等价于 DDPM 的一步加噪。

常见误区

  1. "SDE formulation 是另一种新的扩散模型"。反例:Song 2021 论文里训练的网络跟 DDPM / SMLD 的网络是同一个(NCSN++ / DDPM++ 都用同一架构),只是把训练目标写成连续时间 score matching、把采样写成数值 SDE 积分。FID 提升来自更好的采样器(PC sampler、PF-ODE)和更细的 $\lambda(t)$ weighting,不是来自"新模型"。把 SDE 当模型而非视角,是初学者最大的概念错位。
  2. "VP-SDE 与 VE-SDE 任选一个都行"。反例:VE-SDE 的 $x_T$ 不是单位高斯而是 $\mathcal{N}(x_0, \sigma_T^2 I)$,$\sigma_T$ 在 NCSN 论文中取 50。采样起点必须从这个大方差高斯采(不是 $\mathcal{N}(0, I)$!),否则 score 网络在训练中没见过这种 scale 的输入,直接崩。Stable Diffusion 用 VP(DDPM 系),EDM 用 VE 的变体,这两类不能混用 checkpoint。
  3. "步数越少越好,反正 SDE 可以用高阶 solver"。反例:tradeoff 真实存在——步数越多越接近真 SDE 解,模式覆盖(diversity)越好;步数太少时高阶 solver 仍有较大局部误差。Karras 2022 实测:CIFAR-10 上 18 步 EDM 达 FID 2.0,但 8 步会回到 FID 4+;模式覆盖(Recall 指标)在 SDE 步数充足时显著高于 deterministic ODE。步数 = 多样性 vs 速度的旋钮,不是越少越好。

相关概念

来源

  1. diffusion-principles-vizuara · L8 · Diffusion Models: A Physical Intuition (SDE Framework) — "unifies everything together" · (00:05)
  2. diffusion-principles-vizuara · L8 · 染料扩散物理直觉 + 与 AI 的连接 · (02:06)
  3. diffusion-principles-vizuara · L8 · 布朗运动与微观时间可逆性 · (06:23)
  4. diffusion-principles-vizuara · L8 · "microscopic motions are time reversible" — 整个扩散文献的核心主题 · (08:29)
查看 53 条课程原文出处
CourseLectureTimeExcerpt
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 00: 05 Hello everyone and welcome to the next Hello everyone and welcome to the next lecture of the course principles of lecture of the course principles of lecture of the course principles of diffusion mode…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 01: 09 looked at other lectures. This is looked at other lectures. This is supposed to be a standalone lecture to supposed to be a standalone lecture to supposed to be a standalone lecture to help you get in…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 02: 06 are going to discuss today. are going to discuss today. The good thing is that this unified The good thing is that this unified The good thing is that this unified approach has a lot of connection wit…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 03: 13 Okay. So what do we know about diffusion? In this image diffusion? In this image diffusion? In this image I have shown a dye diffusing or I have shown a dye diffusing or I have shown a dye diffusing o…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 04: 37 Can we or in other words can we recover Can we or in other words can we recover the original structure? the original structure? the original structure? So imagine you're sitting and you see So imagine…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 05: 29 shape of the die. But that is not the case. [snorts] But that is not the case. [snorts] But that is not the case. [snorts] So reversing time is something which is So reversing time is something which …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 06: 23 unpredictable. It's it's very random. It's not moving in a specific direction, It's not moving in a specific direction, It's not moving in a specific direction, but but but they're moving up and down.…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 07: 30 So when the dye was diffusing and when So when the dye was diffusing and when we reversed the whole process, we reversed the whole process, we reversed the whole process, can you tell this movement of…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 08: 29 seconds have passed and someone tells me seconds have passed and someone tells me that Rajett this is the particle which that Rajett this is the particle which that Rajett this is the particle which i…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 09: 29 physics and AI comes into the picture. This is a well-known physical property This is a well-known physical property This is a well-known physical property and it makes sense in when you when we and i…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 10: 33 Okay. Now let us apply this concept Okay. Now let us apply this concept to data. to data. to data. Why are we applying this concept to Why are we applying this concept to Why are we applying this conc…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 11: 30 of them have wide eyes. And the question they ask us is, And the question they ask us is, And the question they ask us is, can you find a distribution can you find a distribution can you find a distri…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 12: 35 when you sample from it, you should get when you sample from it, you should get something which lies within that something which lies within that something which lies within that distribution. distrib…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 13: 28 And to find such distribution we need an algorithm which is clever enough which algorithm which is clever enough which algorithm which is clever enough which can understand that okay uh cats have a ca…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 14: 24 the diffusion process. Now this is where it gets very interesting. In the next it gets very interesting. In the next it gets very interesting. In the next set of slides I'm going to replace the set of…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 15: 37 and then later we recover it. So we destroy something only to recover So we destroy something only to recover So we destroy something only to recover it after some point. it after some point. it after…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 16: 36 visualize the dye being slowly filling visualize the dye being slowly filling the liquid and the structure the liquid and the structure the liquid and the structure disappearing disappearing disappear…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 17: 33 do the opposite. In the reverse process, we will start with noise and then slowly we will start with noise and then slowly we will start with noise and then slowly we will move backwards and we will w…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 18: 33 We have already said that we are going We have already said that we are going to apply the diffusion process to data. to apply the diffusion process to data. to apply the diffusion process to data. Bu…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 19: 41 and you see this motion in a lot of physical processes you see these motions physical processes you see these motions physical processes you see these motions in polymers you see this even in the in p…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 20: 36 On the right hand side we are zooming in on specific particles. So here I have on specific particles. So here I have on specific particles. So here I have taken taken taken seven particles as referenc…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 21: 35 So what am I exactly doing here? Well imagine that instead of these tiny dots imagine that instead of these tiny dots imagine that instead of these tiny dots you have 784 dots and all these dots are y…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 22: 35 I'm planning to take all these images, divide them into pixels and subject each divide them into pixels and subject each divide them into pixels and subject each pixel to a brownian trajectory so that…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 23: 43 two terms a drift term and a noise term. two terms a drift term and a noise term. So DDPM has a specific formulation for So DDPM has a specific formulation for So DDPM has a specific formulation for t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 24: 32 Well, the first thing I can see is that I can obviously see some current lines I can obviously see some current lines I can obviously see some current lines in the river. So I'll say that well in the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 25: 24 a steady drift term which is the current of the lake or the river and the second of the lake or the river and the second of the lake or the river and the second term is the diffusion term which are te…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 26: 20 for this forward diffusion. for this forward diffusion. Okay. So we have converted data to noise Okay. So we have converted data to noise Okay. So we have converted data to noise and the change in the…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 27: 18 You might relate this to a variational You might relate this to a variational autoenccoder where there is an encoder autoenccoder where there is an encoder autoenccoder where there is an encoder and t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 28: 09 with our discussion with the time machine. We were asking that can we machine. We were asking that can we machine. We were asking that can we reverse the reverse the reverse the diffusion of the dye? …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 29: 05 more than 40 years back in the year 1982 by a single author Brian Anderson. by a single author Brian Anderson. by a single author Brian Anderson. And in this paper he wrote a formula And in this paper…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 30: 12 appreciate all the researchers who spend appreciate all the researchers who spend a lot of time in writing these papers. a lot of time in writing these papers. a lot of time in writing these papers. O…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 30: 59 of G of T which is the diffusion term in the forward process. And the second term the forward process. And the second term the forward process. And the second term is the score function. is the score …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 31: 52 reversed from noise to the image and reversed from noise to the image and this is where the learning will happen. this is where the learning will happen. this is where the learning will happen. Why is…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 32: 44 where the density of the sharks is maximum. maximum. maximum. Now score function is exactly like this Now score function is exactly like this Now score function is exactly like this compass and sharks…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 33: 44 density. Now where does the gradient and log come Now where does the gradient and log come Now where does the gradient and log come from? Well, the gradient comes from uh from? Well, the gradient come…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 34: 34 to predict. Remember we started with a discussion that we want to predict the discussion that we want to predict the discussion that we want to predict the distribution of the cats and this distributi…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 35: 34 For now let's simplify things. Let us For now let's simplify things. Let us assume that the score function is assume that the score function is assume that the score function is already learned. alrea…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 36: 30 comprises of the drift in the forward comprises of the drift in the forward diffusion and the factor which depends diffusion and the factor which depends diffusion and the factor which depends on the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 37: 26 ask the question what is the variation ask the question what is the variation in the values of these two points. It in the values of these two points. It in the values of these two points. It turns ou…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 38: 15 in terms of delta t. So if you have delta t you calculate delta x then you delta t you calculate delta x then you delta t you calculate delta x then you again substitute delta t calculate delta again …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 39: 04 differential equation or an SDE. This formulation was laid down in a This formulation was laid down in a This formulation was laid down in a paper by Song and Arman in the year paper by Song and Arman…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 40: 02 represents the noise term. So we go from the first iteration to the second the first iteration to the second the first iteration to the second iteration. Then we again do diffusion iteration. Then we …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 40: 54 you because we know the exact equation of the reverse process and we use that of the reverse process and we use that of the reverse process and we use that same equation for sampling. same equation fo…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 41: 49 data set consisting only of images of data set consisting only of images of handwritten digits which are ones and handwritten digits which are ones and handwritten digits which are ones and eights. ei…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 42: 46 like these handwritten digits. Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal distribution. This is what where we want distribution. …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 43: 41 in the image follows a brownie in motion. motion. Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing in water our in water our in w…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 44: 40 trajectory I have shown but in reality we have these 784 pixels. So it will be we have these 784 pixels. So it will be we have these 784 pixels. So it will be in a 784dimensional space which is in a 7…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 45: 31 Now the next step is we know that the microscopic brownian motion is microscopic brownian motion is microscopic brownian motion is reversible. Uh because as you can see we reversible. Uh because as yo…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 46: 28 as well. And now you can see on the left hand And now you can see on the left hand And now you can see on the left hand side we have noise and we are using the side we have noise and we are using the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 47: 16 diffusion term but this time the trajectory is spread into two very trajectory is spread into two very trajectory is spread into two very nicely and uh we do get a biodal nicely and uh we do get a bio…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 48: 08 have seen this word score function in many papers but only after going through many papers but only after going through many papers but only after going through the SD framework we really understand t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 48: 59 using a very simple approach which can using a very simple approach which can be implemented in Google Collab and uh be implemented in Google Collab and uh be implemented in Google Collab and uh this …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 49: 50 It's a very nicely written paper and after this introduction you should be after this introduction you should be after this introduction you should be able to understand it. Thank you very able to und…