概念 · Module 3: SDE / Score

Probability Flow ODE

— 把 SDE 的布朗项扣掉,留下一条与原 SDE 边缘分布完全一致的确定性 ODE。

反偏见开场:PF-ODE 不是"快一点的 sampler"

很多人第一次见 PF-ODE 都会把它当成"扔掉随机项让采样更快"的工程小聪明。这种理解是错的,而且会把后续好几条性质都看歪。PF-ODE 不是 SDE 的近似,也不是 SDE 的某种极限——它是与 SDE 在每一个时刻 $t$ 上拥有完全相同边缘分布 $p_t$的确定性镜像。两种动力学的样本路径差天差地,但每一帧的群体直方图分毫不差。

把这件事换成物理:SDE 是一群墨水粒子在水里做布朗运动,每颗粒子的具体轨迹随机得无法预测;PF-ODE 是把这群粒子的随机抖动全部抽掉,只保留"使整团墨水的形状与原来一帧帧吻合"的那条平均流动。每颗粒子现在沿一条确定的曲线走——但全体粒子在任意时刻的空间分布,跟原来那群随机抖动的粒子完全一样。

这个"双重身份"是 PF-ODE 全部威力的根。它让你既能享受 SDE 的训练优势——score matching 学的是 $\nabla\log p_t$,跟 SDE / ODE 无关——又能享受 ODE 的推理优势:确定性、可逆、可用高阶 solver、可算似然、可做 image inversion。后面四条性质都是从这一条"边缘等价"派生出来的。

「Today we are going to look at a framework which unifies everything together. ... Both DDPM and score-based approach are special cases of this overall unified approach. ... And when you view everything from this lens, it makes a lot of sense.」
—— Vizuara L8(00:35–01:30)

Vizuara 的 Dr. Raj 这节课开宗明义:DDPM 与 score-based 两条 2020 年前的路线,在 2021 年被 Yang Song 用 SDE 框架统一起来。这个统一里最关键的一块拼图,就是 PF-ODE——它把 SDE 视角下的"随机扩散"和能量视角下的"确定性流"在数学上焊到了一起。

Dr. Raj 还用了一段非常物理的开场:墨水滴入水中扩散,"宏观上时间不可逆——你没法把扩散开的墨水重新聚回原来的球形。但显微镜放大到原子层面,你会发现每颗粒子的布朗运动是可逆的——它在时间反演下分布相同。"这条"微观可逆"性质,正是 PF-ODE 存在的物理直觉来源:随机轨迹在反演下统计上对称,于是那条对应的"平均流动"就可以被严格写出来。

「At the microscopic level, if you look at these individual tiny particles, ... it turns out that these motions are exactly reversible. ... And this is the only theme which is used in the entire diffusion literature in all the different approaches.」
—— Vizuara L8(08:30–09:00)
🟡 教材外补充:Vizuara 没有显式写出 PF-ODE 的 $\tfrac{1}{2} g^2 \nabla\log p_t$ 公式——他在 L8 主要讲 SDE 框架和微观可逆性的物理直觉。CMU L10/L11 则反过来——直接把 PF-ODE 当成既定事实,讨论它怎么和 flow matching、distillation 衔接。这一页要把夹在两门课之间的那道桥补上:把 Vizuara 的物理直觉、Song et al. 2021 的公式、和 CMU 的工程后果(采样步数、flow map)连成一条主线。

定理:每个 SDE 都有一条等价 ODE

Song et al. 2021 的核心定理(论文 Theorem 1 / Eq. 13):给定一个一般的 Itô SDE

$$dx = f(x, t)\, dt + g(t)\, dW,$$

它的解过程 $\{x(t)\}_{t \in [0, T]}$ 的边缘分布 $p_t(x)$ 满足 Fokker–Planck 方程。可以证明,下面这条无随机项的常微分方程

$$\frac{dx}{dt} = f(x, t) - \frac{1}{2}\, g(t)^2\, \nabla_x \log p_t(x),$$

给出的边缘分布与原 SDE 在每个 $t$ 上完全相同。这条 ODE 就是 probability flow ODE(PF-ODE)。注意公式里出现的 score $\nabla_x \log p_t(x)$ ——这正是 score matching 学到的对象,所以你已经训好的 score 网络 $s_\theta(x, t) \approx \nabla_x \log p_t(x)$ 可以直接代进 PF-ODE 用来采样、算似然、反演。

Why 公式长这样:推导走 Fokker–Planck。SDE 的密度演化是 $\partial_t p_t = -\nabla\!\cdot(f p_t) + \tfrac{1}{2} g^2 \Delta p_t$(drift 项 + 扩散项)。把扩散项用恒等式 $\Delta p_t = \nabla\!\cdot(p_t \nabla\log p_t)$ 写成散度形式,整条 FP 方程就变成 $\partial_t p_t = -\nabla\!\cdot\!\big[(f - \tfrac{1}{2}g^2 \nabla\log p_t) p_t\big]$。这是一条纯连续性方程——形如 $\partial_t p + \nabla\!\cdot(pv) = 0$——读出来的有效速度场就是 $v = f - \tfrac{1}{2}g^2 \nabla\log p_t$。这条 ODE 的解流场的边缘密度演化方程,与原 SDE 完全相同。这是连续性方程与 FP 方程之间一个标准的、严格的对应。

关键澄清:边缘等价 $\ne$ 路径等价。边缘等价指 $p_t^{\text{ODE}}(x) = p_t^{\text{SDE}}(x)$ 对所有 $t$ 成立。路径等价才会要求 $x_t^{\text{ODE}} = x_t^{\text{SDE}}$ 对所有 $t$ 都成立——这显然不行,SDE 是随机的,ODE 是确定的。把"边缘等价"和"路径等价"混淆是 PF-ODE 最常见的误解(参见下面的 pitfalls)。

性质一:确定性采样 —— DDIM 就是 PF-ODE 的 Euler

用 PF-ODE 采样的步骤极简单:从 $x_T \sim \mathcal{N}(0, \sigma_T^2 I)$ 起步,用任何 ODE solver 把 $t: T \to 0$ 反着积分一遍,沿途用 $s_\theta(x, t)$ 替换 $\nabla\log p_t$。Euler 一阶离散就是:

$$x_{t - \Delta t} = x_t - \big[f(x_t, t) - \tfrac{1}{2} g(t)^2 s_\theta(x_t, t)\big]\, \Delta t.$$

DDIM(Song, Meng, Ermon 2021)在原论文里是从一个不同的角度——non-Markov forward process——推出来的,但事后大家发现:DDIM 在 VP-SDE 上的更新公式,等价于 PF-ODE 的一阶 Euler 离散。换句话说,DDIM 不是一个全新算法,而是 PF-ODE 在特定 noise schedule 下的具体离散方案。这也解释了为什么 DDIM 在 50 步以下能给出可用的样本,而 DDPM 同等步数下往往糟糕:DDIM 是 ODE 离散,ODE 解的存在唯一性允许大步长;DDPM 是 SDE 离散,每一步都重新注入噪声,大步长会让噪声主导信号。

Why 这件事重要:在 DDIM 出现之前,"扩散模型采样要 1000 步"是默认事实。DDIM 把步数砍到 50;后续 DPM-Solver、Heun、RK4 等高阶 PF-ODE solver 把步数砍到 10–20;distillation / consistency models 进一步砍到 1–4 步。所有这些采样加速的根都是同一句话:"采样不必走 SDE,可以走与之边缘等价的 ODE。" 没有 PF-ODE,就没有上面整条加速线。

另一条值得强调的工程线索:DDIM 之所以能脱离 SDE 框架成立,正是因为 PF-ODE 存在。如果没有"边缘等价的 ODE"这条定理,2021 年的研究者面对 50 步采样需求时大概只能继续在 SDE 框架里凑数值技巧。PF-ODE 把整个问题从随机过程数值分析(一个相对小众、工具不够成熟的领域)转移到了常微分方程数值分析(一个 50 年来积累极厚的领域),整条采样加速产业链由此打开。这是为什么"PF-ODE 只是个理论玩具"的判断是完全错误的——它是 2021 年之后整个扩散模型工程加速浪潮的理论触发器。

性质二:精确似然 —— Hutchinson trace 把行列式打掉

PF-ODE 是一种 continuous normalizing flow(CNF),所以满足瞬时变量替换公式:

$$\frac{d\log p_t(x(t))}{dt} = -\,\mathrm{tr}\!\left(\nabla_x v_\theta(x, t)\right), \quad v_\theta := f - \tfrac{1}{2} g^2 s_\theta.$$

把 $t$ 从 $0$ 积到 $T$,加上 $\log p_T(x_T)$(高斯,闭式),就得到数据点 $x_0$ 的精确对数似然:

$$\log p_0(x_0) = \log p_T(x_T) + \int_0^T \mathrm{tr}(\nabla v_\theta)\, dt.$$

高维下 $\mathrm{tr}(\nabla v_\theta)$ 不能显式算 $d$ 个对角元($d=10^6$ 量级),但 Hutchinson 随机迹估计 $\mathrm{tr}(A) = \mathbb{E}_\epsilon[\epsilon^\top A \epsilon]$ 只需要一次 Jacobian–向量积,$O(d)$。这是 Song et al. 2021 在 CIFAR-10 上首次用 score-based 模型刷出 NLL SOTA的工程关键——在 PF-ODE 出现之前,diffusion 模型只能给出 ELBO 下界,没法直接报告 log-likelihood。

性质三:图像 inversion —— ODE 可逆,SDE 不行

给一张真实图 $x_0$,想找出"哪个 $x_T$ 经过这个模型会生成它?"——这就是 image inversion,是 prompt editing、stylization、SDEdit 等下游应用的基础。在 PF-ODE 框架下做法极简单:把求解器反向跑,$t: 0 \to T$ 积分同一条 ODE,就能从 $x_0$ 唯一地反推到 $x_T$。这是因为 ODE 解的存在唯一性定理保证了双射。

SDE 没法这么干——SDE 是随机的,同一个 $x_0$ 反过来跑可以对应无穷多种 $x_T$ 轨迹,没有唯一的 inverse。要在 SDE 框架下做 inversion 只能近似(比如 SDEdit 加噪声再去噪声、DDIM inversion 借 PF-ODE 这条捷径)。实际工业 pipeline 里所有 inversion 算法走的都是 PF-ODE 这条线,没有例外。

性质四:高阶 solver —— DPM-Solver、Heun、RK4

PF-ODE 是一条"普通的"常微分方程,所有 50 年来积累的 ODE 数值积分文献都适用。Heun(二阶 Runge–Kutta)一步两次 score evaluation,但精度从 $O(\Delta t)$ 升到 $O(\Delta t^2)$;DPM-Solver 利用 score 的半解析结构进一步压低误差;RK4 / DPM-Solver++ 把 SOTA 步数压到 10 步以内。这条工程线在 SDE 框架下几乎不可能存在——SDE 的高阶 solver(Milstein、Euler–Maruyama 改进版)一来计算复杂,二来误差结构和 ODE 完全不同,三来扩散模型上根本没人去做。"高阶 solver = 更少 NFE = 更快采样"这条产业链完全建立在 PF-ODE 之上。

SDE vs PF-ODE:随机性如何换步数

既然两条动力学边缘等价,凭什么用哪一条?经验事实:

  • SDE 路径:多样性高、对模型误差更鲁棒(每步注入的噪声会"洗掉"上一步累积的 score 估计误差),但需要的步数多(500–1000 步)。
  • PF-ODE 路径:步数少(10–50 步即可),但确定性轨迹会把 score 估计误差一路累积下去;多样性也更窄——同一个起点 $x_T$ 始终给同一个 $x_0$。

EDM(Karras et al. 2022)做了系统性消融:在同一份训练好的模型上,SDE 采样的 FID 通常比 PF-ODE 略好(多样性优势),但 NFE 至少需要 4–8 倍。工业部署普遍选 PF-ODE + 高阶 solver。

连接:PF-ODE 是一种 CNF

PF-ODE = CNF(用 score-matching 训练)。一个 CNF $\dfrac{dx}{dt} = v_\theta(x, t)$ 可以用最大似然训(经典 Neural ODE 路线,慢)、用 flow matching 训(FM 路线,快)、或者用 score matching 间接训——这第三条就是 diffusion / PF-ODE 路线。Score 网络 $s_\theta$ 学到 $\nabla\log p_t$ 之后,速度场 $v_\theta = f - \tfrac{1}{2}g^2 s_\theta$ 就构造出来了,不需要直接学。这就是为什么 diffusion 与 flow matching 在 CNF 框架下被视为同一种生成模型的两种训练 shortcut。

这条桥的工程后果非常具体:Stable Diffusion 3、SORA、Flux 这些"新一代"模型表面上是 flow matching,实际上跑的是 CNF;老一代 Stable Diffusion 1/2、DALL·E 2、Imagen 是 diffusion,实际上跑的也是 CNF(PF-ODE 形式)。两代之间的差别不是模型架构,而是怎么训那个共享的速度场。这也是为什么 CMU L10 / L11 在讨论 distillation 和 flow maps 时可以把 diffusion 和 flow matching 一起讨论——它们的推理时动力学(一条 ODE)是统一的。

「At inference time they solve the so-called probability flow ODE ... and we argue that diffusion and flow matching are not optimal in utilizing the network capacity — because even if your network predicts $u_t$ perfectly at every single point, you will still need many many steps to simulate the ODE accurately.」
—— CMU 10799 L11(Linqi Zhou, Luma AI,04:13–04:50)

Luma AI 的 Linqi Zhou 在这段话里把 PF-ODE 的工程瓶颈点了出来:哪怕 score / velocity 学到了完美值,积分这条 ODE 本身还是需要不少步——这是 ODE 数值积分的内在限制,不是模型不够准。这就是为什么后续会出现 consistency models、flow maps、shortcut models 这一整条"绕开 ODE 积分"的研究线。换言之,PF-ODE 把扩散模型的瓶颈从"训练目标定义不清"(DDPM 时代的 ELBO 推导冗长)转化为"推理时数值积分需要多少步"——前者是数学问题,后者是数值分析问题,后者明显更容易被工程优化攻克。

把这条思路再推一步:consistency models(Song et al. 2023)做的事情,本质是把 PF-ODE 的多步积分蒸馏成一步映射——它训一个网络 $f_\theta(x_t, t)$,让它对 PF-ODE 轨迹上的任意点都映回同一个 $x_0$。这等价于学一条"flow map",把 ODE 的 50 步压缩到 1 步。Rectified Flow(Liu et al. 2022)也是同一思路的变体:用线性插值预设路径,让训出的 PF-ODE 在直线上跑得更直,于是大步长 Euler 误差更小。所有这些"采样加速派"工作,都把 PF-ODE 作为基线和理论锚点——优化的目标是"逼近 PF-ODE 的 $x_0$,但用更少步数"。

「If you're doing flow matching, then we can also learn flow maps for the probability flow ODE or the flow matching ODE — essentially a flow map is just a function to tell you to jump from $x_t$ at time $t$ to $x_s$ at time $s$.」
—— CMU 10799 L10(50:04–50:40)

主图:SDE 路径 vs PF-ODE 路径

$x_T$(同一起点) $t=0$ 数据 $x_T$ SDE: 3 个不同终点 PF-ODE: 1 个确定终点 在每个时刻 $t$,两种动力学的群体直方图 $p_t(x)$ 相同——但单条轨迹完全不同。

蓝色三条波折线是同一起点下 SDE 跑三次的随机轨迹,终点各不相同;橙色实线是同一起点下 PF-ODE 的唯一确定轨迹。视觉冲击点:起点相同的情况下,SDE 轨迹"撒得开"——这是布朗项 $g(t)\,dW$ 在做工;PF-ODE 一条线落地——确定性 ODE 的双射性质。但——这是 PF-ODE 最反直觉的地方——把蓝色"群体"和橙色"群体"分别在某个 $t$ 切一刀看横截面直方图,两者完全重合。

下面四个按钮切换故事的四帧:① 同一个 $x_T$ 起点;② SDE 跑出多条随机轨迹;③ PF-ODE 跑出一条确定轨迹;④ 把两种群体在 $t=0$ 时的边缘分布叠加——重合。不变量:每个 $t$ 上的边缘分布相同;变化的是路径形态。

$x_T$ 单个起点,看接下来怎么走。
$x_T \sim \mathcal{N}(0, \sigma_T^2 I)$:同一个噪声起点。

手算:VP-SDE 上的 PF-ODE 一步 Euler

取 1-D VP-SDE:$f(x, t) = -\tfrac{1}{2}\beta(t) x$,$g(t) = \sqrt{\beta(t)}$。设 $\beta(t) = 0.1$(常数 noise schedule,简化),当前状态 $x_t = 1.5$,已训好的 score 网络给出 $s_\theta(x_t, t) = -0.8$。代入 PF-ODE:

$$\frac{dx}{dt} = -\tfrac{1}{2}(0.1)(1.5) - \tfrac{1}{2}(0.1)(-0.8) = -0.075 + 0.04 = -0.035.$$

用 $\Delta t = 0.1$ 反向积分(采样方向是 $t \to t - \Delta t$,所以走 $-\Delta t$ 实际上对应往数据靠近):

$$x_{t - 0.1} \approx x_t - (-0.035)(0.1) = 1.5 + 0.0035 = 1.5035.$$

(VP-SDE 的反向时间方向上,更新公式是 $x_{t-\Delta t} = x_t + [\ldots]\Delta t$。)对比 SDE Euler–Maruyama 同样步长下还要加上 $\sqrt{\beta \Delta t}\,\epsilon$,其中 $\epsilon \sim \mathcal{N}(0, 1)$ —— 即额外 $\pm 0.1$ 量级的随机扰动。两条更新的期望相同(都是 $1.5035$),但 SDE 每次撒出去一个噪声,PF-ODE 不撒。把这一步反复迭代上千次,就能看到群体分布相同、单条轨迹截然不同的现象。

三大常见误区

  1. "PF-ODE 是 SDE 的近似——边缘只是大致相同。" 反例:Song et al. 2021 Theorem 1 给的是等式,不是近似。$p_t^{\text{ODE}}(x) = p_t^{\text{SDE}}(x)$ 在所有 $t$、所有 $x$ 上严格成立,证明走 Fokker–Planck 方程的恒等变形——把扩散项 $\tfrac{1}{2}g^2 \Delta p$ 改写成 $\nabla\!\cdot(\tfrac{1}{2}g^2 p \nabla\log p)$,这是恒等变换不是近似。实务上 PF-ODE 的样本质量和 SDE 不完全一样,差别来自数值积分误差和有限步数 score 估计误差,不是定理本身的近似。
  2. "PF-ODE 是 SDE 的 limit(让噪声系数趋近 0)。" 反例:让 $g \to 0$ 会同时去掉随机项和 score 项前的 $\tfrac{1}{2}g^2$ 系数——剩下来只有 drift $f$,那是一条错误的 ODE,边缘分布和原 SDE 完全不一样(会塌缩到 drift 决定的确定性流,丢失全部噪声扩散信息)。PF-ODE 的正确推导不是取 $g \to 0$,而是保留 $g$ 不变,但用 FP 方程的恒等变形把扩散项吸进一个等效速度场。PF-ODE 和原 SDE 用的是同一个 $g$、同一个 score。
  3. "DDIM 是 DDPM 的某种近似或简化。" 反例:DDIM 在 VP-SDE 上的更新公式严格等于 PF-ODE 的 Euler 一阶离散——这是一条严格的对应(参 Song, Meng, Ermon 2021 附录、以及 Salimans & Ho 2022 的细致推导)。DDPM 是 SDE 的 Euler–Maruyama 离散,DDIM 是 PF-ODE 的 Euler 离散。两者不是"DDIM = DDPM 的快速版"那种近似关系,而是两种不同动力学(同边缘)的两种 Euler 离散。DDIM 在 50 步以下能稳定工作,根因是 ODE 解的稳定性允许大步长——这是 ODE 数值分析的常识,不是 DDIM 算法本身的魔法。把误区反过来理解会更清晰:DDIM 论文里写的"non-Markov forward process"是 2020 年的事后推导,2021 年大家在 PF-ODE 框架下回头看,才发现它就是这条 ODE 的 Euler 离散。两条推导路径在数学上完全等价,但 PF-ODE 视角更通用——它允许直接换上 Heun、RK4、DPM-Solver 等高阶 solver,而 DDIM 原始推导框架下做不到这一点。

相关概念

位置关系:PF-ODE 处在 SDE(Module 3)与 CNF / Flow Matching(Module 4)之间的桥梁位置。从左侧看,它是 SDE 的边缘镜像;从右侧看,它是用 score-matching 间接训练的 CNF。这条桥让 diffusion 和 flow matching 在推理时是同一种动力学(都是 ODE 积分),区别只在训练时损失函数的选择。
  • SDE Formulation — PF-ODE 的"原生"动力学;PF-ODE 是它的确定性镜像。
  • Score Function — PF-ODE 速度场的核心可学习对象 $\nabla\log p_t$。
  • Continuous Normalizing Flow — PF-ODE 在结构上就是一种 CNF,区别只是训练损失。
  • DDIM — PF-ODE 的一阶 Euler 离散;50 步以下采样的标准方案。
  • Flow Matching — 与 PF-ODE 推理动力学相同、训练方式不同的同族方法。
  • Consistency Models — 通过 distillation 学一个 flow map,绕开 PF-ODE 的多步积分。

来源

  1. diffusion-principles-vizuara · L8 · 「framework which unifies everything together」 · (00:35)
  2. diffusion-principles-vizuara · L8 · 「microscopic motions are exactly reversible」 · (08:30)
  3. diffusion-principles-vizuara · L8 · 「Brownian motion / 微观可逆性」 · (06:24)
  4. cmu-10799-diffusion-flow · L10 · 「flow maps for probability flow ODE」 · (50:04)
  5. cmu-10799-diffusion-flow · L11 (Linqi Zhou, Luma AI) · 「at inference time they solve the probability flow ODE」 · (04:13)
  6. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. ICLR. — Theorem 1, Eq. 13(PF-ODE 公式与边缘等价证明)。
查看 55 条课程原文出处
CourseLectureTimeExcerpt
cmu-10799-diffusion-flow L10: CMU 10799 S26: Lecture 11 - Guest Lecture Linqi (Alex) Zhou from Luma AI - Diffusion & Flow Matching 04: 13 and uh ultimately the loss comes down to a simple L2 loss uh which is extremely a simple L2 loss uh which is extremely a simple L2 loss uh which is extremely stable and very scalable to large number s…
cmu-10799-diffusion-flow L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching 50: 04 OD or PD or whatever um and then the flow maps because we also look at ODEs flow maps because we also look at ODEs flow maps because we also look at ODEs right especially if If you're doing flow right…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 00: 05 Hello everyone and welcome to the next Hello everyone and welcome to the next lecture of the course principles of lecture of the course principles of lecture of the course principles of diffusion mode…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 01: 09 looked at other lectures. This is looked at other lectures. This is supposed to be a standalone lecture to supposed to be a standalone lecture to supposed to be a standalone lecture to help you get in…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 02: 06 are going to discuss today. are going to discuss today. The good thing is that this unified The good thing is that this unified The good thing is that this unified approach has a lot of connection wit…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 03: 13 Okay. So what do we know about diffusion? In this image diffusion? In this image diffusion? In this image I have shown a dye diffusing or I have shown a dye diffusing or I have shown a dye diffusing o…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 04: 37 Can we or in other words can we recover Can we or in other words can we recover the original structure? the original structure? the original structure? So imagine you're sitting and you see So imagine…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 05: 29 shape of the die. But that is not the case. [snorts] But that is not the case. [snorts] But that is not the case. [snorts] So reversing time is something which is So reversing time is something which …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 06: 23 unpredictable. It's it's very random. It's not moving in a specific direction, It's not moving in a specific direction, It's not moving in a specific direction, but but but they're moving up and down.…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 07: 30 So when the dye was diffusing and when So when the dye was diffusing and when we reversed the whole process, we reversed the whole process, we reversed the whole process, can you tell this movement of…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 08: 29 seconds have passed and someone tells me seconds have passed and someone tells me that Rajett this is the particle which that Rajett this is the particle which that Rajett this is the particle which i…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 09: 29 physics and AI comes into the picture. This is a well-known physical property This is a well-known physical property This is a well-known physical property and it makes sense in when you when we and i…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 10: 33 Okay. Now let us apply this concept Okay. Now let us apply this concept to data. to data. to data. Why are we applying this concept to Why are we applying this concept to Why are we applying this conc…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 11: 30 of them have wide eyes. And the question they ask us is, And the question they ask us is, And the question they ask us is, can you find a distribution can you find a distribution can you find a distri…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 12: 35 when you sample from it, you should get when you sample from it, you should get something which lies within that something which lies within that something which lies within that distribution. distrib…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 13: 28 And to find such distribution we need an algorithm which is clever enough which algorithm which is clever enough which algorithm which is clever enough which can understand that okay uh cats have a ca…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 14: 24 the diffusion process. Now this is where it gets very interesting. In the next it gets very interesting. In the next it gets very interesting. In the next set of slides I'm going to replace the set of…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 15: 37 and then later we recover it. So we destroy something only to recover So we destroy something only to recover So we destroy something only to recover it after some point. it after some point. it after…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 16: 36 visualize the dye being slowly filling visualize the dye being slowly filling the liquid and the structure the liquid and the structure the liquid and the structure disappearing disappearing disappear…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 17: 33 do the opposite. In the reverse process, we will start with noise and then slowly we will start with noise and then slowly we will start with noise and then slowly we will move backwards and we will w…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 18: 33 We have already said that we are going We have already said that we are going to apply the diffusion process to data. to apply the diffusion process to data. to apply the diffusion process to data. Bu…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 19: 41 and you see this motion in a lot of physical processes you see these motions physical processes you see these motions physical processes you see these motions in polymers you see this even in the in p…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 20: 36 On the right hand side we are zooming in on specific particles. So here I have on specific particles. So here I have on specific particles. So here I have taken taken taken seven particles as referenc…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 21: 35 So what am I exactly doing here? Well imagine that instead of these tiny dots imagine that instead of these tiny dots imagine that instead of these tiny dots you have 784 dots and all these dots are y…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 22: 35 I'm planning to take all these images, divide them into pixels and subject each divide them into pixels and subject each divide them into pixels and subject each pixel to a brownian trajectory so that…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 23: 43 two terms a drift term and a noise term. two terms a drift term and a noise term. So DDPM has a specific formulation for So DDPM has a specific formulation for So DDPM has a specific formulation for t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 24: 32 Well, the first thing I can see is that I can obviously see some current lines I can obviously see some current lines I can obviously see some current lines in the river. So I'll say that well in the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 25: 24 a steady drift term which is the current of the lake or the river and the second of the lake or the river and the second of the lake or the river and the second term is the diffusion term which are te…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 26: 20 for this forward diffusion. for this forward diffusion. Okay. So we have converted data to noise Okay. So we have converted data to noise Okay. So we have converted data to noise and the change in the…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 27: 18 You might relate this to a variational You might relate this to a variational autoenccoder where there is an encoder autoenccoder where there is an encoder autoenccoder where there is an encoder and t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 28: 09 with our discussion with the time machine. We were asking that can we machine. We were asking that can we machine. We were asking that can we reverse the reverse the reverse the diffusion of the dye? …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 29: 05 more than 40 years back in the year 1982 by a single author Brian Anderson. by a single author Brian Anderson. by a single author Brian Anderson. And in this paper he wrote a formula And in this paper…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 30: 12 appreciate all the researchers who spend appreciate all the researchers who spend a lot of time in writing these papers. a lot of time in writing these papers. a lot of time in writing these papers. O…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 30: 59 of G of T which is the diffusion term in the forward process. And the second term the forward process. And the second term the forward process. And the second term is the score function. is the score …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 31: 52 reversed from noise to the image and reversed from noise to the image and this is where the learning will happen. this is where the learning will happen. this is where the learning will happen. Why is…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 32: 44 where the density of the sharks is maximum. maximum. maximum. Now score function is exactly like this Now score function is exactly like this Now score function is exactly like this compass and sharks…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 33: 44 density. Now where does the gradient and log come Now where does the gradient and log come Now where does the gradient and log come from? Well, the gradient comes from uh from? Well, the gradient come…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 34: 34 to predict. Remember we started with a discussion that we want to predict the discussion that we want to predict the discussion that we want to predict the distribution of the cats and this distributi…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 35: 34 For now let's simplify things. Let us For now let's simplify things. Let us assume that the score function is assume that the score function is assume that the score function is already learned. alrea…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 36: 30 comprises of the drift in the forward comprises of the drift in the forward diffusion and the factor which depends diffusion and the factor which depends diffusion and the factor which depends on the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 37: 26 ask the question what is the variation ask the question what is the variation in the values of these two points. It in the values of these two points. It in the values of these two points. It turns ou…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 38: 15 in terms of delta t. So if you have delta t you calculate delta x then you delta t you calculate delta x then you delta t you calculate delta x then you again substitute delta t calculate delta again …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 39: 04 differential equation or an SDE. This formulation was laid down in a This formulation was laid down in a This formulation was laid down in a paper by Song and Arman in the year paper by Song and Arman…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 40: 02 represents the noise term. So we go from the first iteration to the second the first iteration to the second the first iteration to the second iteration. Then we again do diffusion iteration. Then we …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 40: 54 you because we know the exact equation of the reverse process and we use that of the reverse process and we use that of the reverse process and we use that same equation for sampling. same equation fo…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 41: 49 data set consisting only of images of data set consisting only of images of handwritten digits which are ones and handwritten digits which are ones and handwritten digits which are ones and eights. ei…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 42: 46 like these handwritten digits. Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal distribution. This is what where we want distribution. …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 43: 41 in the image follows a brownie in motion. motion. Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing in water our in water our in w…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 44: 40 trajectory I have shown but in reality we have these 784 pixels. So it will be we have these 784 pixels. So it will be we have these 784 pixels. So it will be in a 784dimensional space which is in a 7…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 45: 31 Now the next step is we know that the microscopic brownian motion is microscopic brownian motion is microscopic brownian motion is reversible. Uh because as you can see we reversible. Uh because as yo…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 46: 28 as well. And now you can see on the left hand And now you can see on the left hand And now you can see on the left hand side we have noise and we are using the side we have noise and we are using the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 47: 16 diffusion term but this time the trajectory is spread into two very trajectory is spread into two very trajectory is spread into two very nicely and uh we do get a biodal nicely and uh we do get a bio…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 48: 08 have seen this word score function in many papers but only after going through many papers but only after going through many papers but only after going through the SD framework we really understand t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 48: 59 using a very simple approach which can using a very simple approach which can be implemented in Google Collab and uh be implemented in Google Collab and uh be implemented in Google Collab and uh this …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 49: 50 It's a very nicely written paper and after this introduction you should be after this introduction you should be after this introduction you should be able to understand it. Thank you very able to und…