概念 · Module 3 Score 视角

Langevin Dynamics

— 沿 score 漂移加高斯抖动,让粒子最终按 $p(x)$ 分布。

很多人第一次看到 Langevin 的更新式 $x \leftarrow x + \frac{\delta}{2}\nabla \log p + \sqrt{\delta}\,\epsilon$,第一反应是:「这不就是带噪声的 SGD 吗?」不是。这是初学者最致命的误读。SGD 的目标是把 loss 推到极小,参数最后稳定下来;Langevin 的目标恰恰相反——让粒子永远不要停下,使得长时间下来访问每个点 $x$ 的频率正好等于密度 $p(x)$。一个是优化、一个是采样,是两件完全不同的任务,只是更新式长得像而已。

把 Langevin 看成「加噪 SGD」的人,会犯三类典型错:调小步长想让它「更准确」(结果完全失去 mixing)、跑到 score 接近零就停下(停下意味着只采到一个 mode 的样本)、以及奇怪为什么训练好的 score 网络直接跑 Langevin 会生成噪声(因为流形外 score 估不准,需要 annealed 调度)。下面三个 pitfall 会逐一给反例。先把正确的图像建立起来。

Vizuara 的 Dr. Raj 在 Lecture 5《Energy Based Models / Score Function》给的物理图像最直观:把 $-\log p(x)$ 想象成一个势能面 $V(x) = -\log p(x)$,山谷(势能低)就是高概率区(density 高),山顶(势能高)就是低概率区。Langevin 就是一颗布朗粒子被扔进这个势能面,受两种力同时作用——

「Score $\nabla\log p$ 就是势能的负梯度 $-\nabla V$,它告诉粒子往哪里『下坡』。但如果你只听 score、不加噪声,粒子会卡死在距离起点最近的那个山谷里,永远到不了别的山谷。加噪声 $\sqrt{\delta}\epsilon$ 的意义,是让粒子有概率翻过低矮的山脊,去到分布的其他 mode。drift 与 diffusion 的平衡,让稳态访问频率正好是 $p(x)$。」—— Vizuara L5 主旨改写

这个 Brownian-particle-in-potential 的画面是物理学家 Paul Langevin 1908 年提出的,原本用来描述悬浮颗粒在热涨落下的运动。机器学习借用它的数学结构来做采样:势能定义成 $-\log p$,温度调成 1,粒子的稳态分布就是 $p$ 本身。这不是巧合——Fokker-Planck 方程给的是严格的「连续时间下的精确收敛」保证。

CMU 10-799 Lecture 4《Score-based Models》在另一个角度补完图像:在扩散语境里,Langevin 不是一个独立工具,而是反向去噪过程的原子步。DDPM 的反向链 $x_T \to x_{T-1} \to \dots \to x_0$,每一步「去一点噪声」的操作,重写出来正好就是「在 noise level $\sigma_t$ 下做一次 Langevin step」。两门课从两个不同的起点出发(Vizuara 从 EBM/score 切入,CMU 从扩散反向过程切入),最后会合在同一个公式上——这是 score-based 范式的全部魅力所在。

在训练里,这件事对应一个非常具体的工程问题:你训了一个 score network $s_\theta(x) \approx \nabla \log p(x)$(怎么训留给 Score Matching 那一页),怎么用它生成新样本? 答案就是 Langevin——把 $\nabla \log p$ 换成 $s_\theta$,从一个初始噪声 $x_0 \sim \mathcal{N}(0, I)$ 开始,迭代几百到几千步,最后粒子的位置就是 $p_{\text{data}}$ 的一个样本。这就是 Song & Ermon 2019 的 NCSN,以及后续整个 score-based diffusion 家族的生成机制。

所以一句话总结这个开场:Langevin 不是优化、不是「噪声 SGD」,它是用 score 把粒子的访问频率塑造成目标分布的一台机器。drift 项决定密度的形状,noise 项决定混合的强度,两者通过 Fokker-Planck 的精确平衡,让稳态等于 $p$。理解这一点之后,后面所有公式(DDPM 反向、SDE 框架、Probability Flow ODE)都只是这个核心思想在不同坐标下的表达。

1. 从 SDE 到离散更新

连续时间下,Langevin 写成一个随机微分方程(SDE):

$$ dx_t = \nabla_x \log p(x_t)\,dt + \sqrt{2}\,dW_t $$

这里 $W_t$ 是标准布朗运动,$dW_t$ 在直觉上等于 $\sqrt{dt}\cdot \mathcal{N}(0, I)$(这是布朗运动方差随时间线性增长的核心性质)。第一项是漂移项 (drift),把粒子推向 $p$ 的高密度区;第二项是扩散项 (diffusion),不断撒入高斯噪声。两项平衡时,粒子分布会收敛到 $p$ 本身。

Why 系数是 $\sqrt{2}$?不是任意挑的。Fokker-Planck 方程描述了粒子分布 $p_t(x)$ 随时间的演化:对于一般 SDE $dx = \mu(x)dt + \sigma\,dW$,有 $\partial_t p_t = -\nabla\cdot(\mu p_t) + \frac{1}{2}\sigma^2 \Delta p_t$。代入 $\mu = \nabla\log p$、$\sigma = \sqrt{2}$,得 $\partial_t p_t = -\nabla\cdot(p_t \nabla\log p) + \Delta p_t$。当 $p_t = p$(稳态),右边两项完全相消(用恒等式 $p\nabla\log p = \nabla p$ 验证),所以 $\partial_t p = 0$,$p$ 确实是不变测度。如果系数不是 $\sqrt{2}$,温度就会"不对",稳态分布会变成 $p^\beta$ 而不是 $p$。

实践中我们不能跑连续 SDE,必须离散化。最简单的欧拉-丸山(Euler-Maruyama)离散化,步长 $\delta$(讲师有时也写 $\eta$ 或 $\epsilon$,不同 paper 不一致,注意区分):

$$ x_{k+1} = x_k + \frac{\delta}{2}\,\nabla_x \log p(x_k) + \sqrt{\delta}\,\epsilon_k, \quad \epsilon_k \sim \mathcal{N}(0, I) $$

Why drift 配 $\frac{\delta}{2}$、noise 配 $\sqrt{\delta}$?因为 $dt$ 离散成 $\delta$ 后,drift 项的尺度是 $\delta$,而布朗运动增量 $dW$ 的尺度是 $\sqrt{dt} = \sqrt{\delta}$(方差线性、标准差平方根)。$\sqrt{2}$ 在离散下被「分裂」:$\sqrt{2}\cdot\sqrt{\delta} = \sqrt{2\delta}$,配套地 drift 写成 $\frac{2\delta}{2}\nabla\log p$,把那个 2 移到分母。这样写出来 $(\delta/2,\ \sqrt{\delta})$ 是对称的、好记的形式,等价于 $(\delta,\ \sqrt{2\delta})$ 但更通用——Welling & Teh 2011(SGLD 论文)就是用前一种。两种你都会看到,记住方差与步长的比例关系就行:方差是步长的一次方,标准差是平方根。

2. 为什么必须有 score(也就是为什么 score-based 模型重要)

更新式里出现的不是 $p$、不是 $\log p$,而是 $\nabla \log p$——对数密度的梯度。这个看似细小的区别,是 Langevin 跟 score-based modeling 的命脉所在:

  • 把 $p$ 写成能量形式 $p(x) = \frac{1}{Z}\exp(-E(x))$,其中 $Z = \int \exp(-E(x))dx$ 是归一化常数。在高维下 $Z$ 根本算不出来(积分维数太高、没有解析形式)——这是 EBM 几十年来的核心痛点。
  • 但是!$\nabla \log p = \nabla(-E - \log Z) = -\nabla E$。因为 $Z$ 跟 $x$ 无关,$\nabla \log Z = 0$。score 自动绕过了归一化常数。
  • 所以只要学到 $s_\theta(x) \approx \nabla \log p(x)$(用 score matching 或 denoising score matching,损失函数不依赖 $Z$),就能跑 Langevin 采样。这正是 Hyvärinen 2005 → Vincent 2011 → Song & Ermon 2019 这条线的核心发现。

反过来:如果我们必须用 $p$ 本身(而不是 score)来采样,那就只能走 MCMC 的 Metropolis-Hastings 路线,需要算 $p(x')/p(x)$ 这种比值——比值中 $Z$ 会消掉,但 $E(x')$ 和 $E(x)$ 都得算,仍然要 train 一个能算能量的模型。score 路线更纯粹:网络直接吐 $\nabla\log p$,连 $E$ 都不需要建模。

势能 $V(x) = -\log p(x)$(山谷 = 高概率) mode 1 mode 2 山脊 · 低 $p$ drift · $\nabla\log p$ noise · $\sqrt{\delta}\epsilon$ 无噪声 → 卡在 mode 1;有噪声 → 偶尔翻山到 mode 2,访问频率 ∝ $p$

3. Annealed Langevin —— 工程上必须做的修正

到这里你可能以为:训好 score 网络,跑 Langevin,搞定。不行。Song & Ermon 在 NCSN 论文里用实验证明:直接对原始数据分布 $p_{\text{data}}$ 学 score、然后跑 vanilla Langevin,几乎一定失败。原因有二:

  • 流形假说问题:真实数据(图像、音频)集中在一个低维流形上,流形外几乎所有点 $p_{\text{data}}(x) \approx 0$,score $\nabla\log p$ 数值上未定义、网络也学不到东西。但 Langevin 初始化在 $\mathcal{N}(0, I)$,第一步就在流形外面——网络给出垃圾 score,迭代直接发散或卡死。
  • 低密度区域问题:即使所有 mode 之间有微弱通路,分隔它们的「山脊」概率极低、score 极小、加上的噪声不足以翻过去——粒子会被困在初始化附近的局部 mode,没法 mixing。

修正方案:annealed Langevin。先对数据分布做大噪声平滑得到 $p_{\sigma_1}(x) = \int p_{\text{data}}(y) \mathcal{N}(x; y, \sigma_1^2 I)dy$,$\sigma_1$ 取得很大(比如 $\sigma_1 = 50$),让原本相互分隔的 modes「糊」成一片连续的分布。在这个平滑分布上跑几步 Langevin,得到一个粗糙样本。然后切换到稍小的 $\sigma_2 < \sigma_1$,跑几步,再切到更小的 $\sigma_3$,直到 $\sigma_L$ 接近 0,最后那个分布就是 $p_{\text{data}}$ 本身。整条链 $L$ 一般是 10–1000 个 noise level,每个 level 跑 5–100 步 Langevin。

更深刻的洞察:这个 noise-schedule 其实就是温度退火。高 $\sigma$ 等价于「高温」——粒子能轻松翻山;低 $\sigma$ 等价于「低温」——粒子精修到准确位置。直接低温启动会卡 mode;先高温再退火(annealing),是 statistical physics 里几十年来对付多模分布的标准把戏。

🟡 教材外补充:DDPM 反向过程 ≈ annealed Langevin。这是把两条历史线缝起来的关键。DDPM(Ho et al. 2020)的反向更新原本写成 $x_{t-1} = \frac{1}{\sqrt{\alpha_t}}(x_t - \frac{1-\alpha_t}{\sqrt{1-\bar\alpha_t}}\epsilon_\theta(x_t, t)) + \sigma_t z$,看起来跟 Langevin 没关系。但 Tweedie 恒等式给出 $\epsilon_\theta = -\sqrt{1-\bar\alpha_t}\cdot s_\theta$,代回去化简:$x_{t-1} \approx x_t + c_1(t)\,s_\theta(x_t, t) + c_2(t)\,z$,正好是 noise level $t$ 上的一步 Langevin。Yang Song 2021 的「Score SDE」论文把这件事在连续时间下正式化:DDPM 的离散反向链 = reverse-time SDE 的特定离散,而 reverse-time SDE 与 annealed Langevin 共享同一条Probability Flow ODE。从此 Ho 2020 的 "diffusion model" 和 Song 2019 的 "score-based model" 不再是两个东西。

4. 离散化误差与 MH 修正

Langevin SDE 在连续时间下严格收敛到 $p$(Fokker-Planck 证完事),但欧拉离散后有 $O(\delta^2)$ 的 bias——稳态分布会偏移一点点,不再是精确的 $p$。两种处理:

  • 取小 $\delta$。bias 随 $\delta$ 线性下降,但代价是需要更多步数才能 mixing。实践中 $\delta \in [10^{-5}, 10^{-3}]$,跑 1000–10000 步。这是 score-based diffusion 实际生成图像所用的成本量级——也是为什么 diffusion 推理慢的根本原因(每张图要几百到几千次 score 网络前向)。
  • Metropolis-Hastings 修正,得到 MALA(Metropolis-Adjusted Langevin Algorithm)。每步先按 Langevin 提议 $x'$,再以接受概率 $\min(1, \frac{p(x')q(x|x')}{p(x)q(x'|x)})$ 决定是否接受。理论上 MH 完全消除离散 bias,稳态严格是 $p$。代价:需要算 $p(x')/p(x)$ 这个比值——而 score-based 模型只学了 $\nabla\log p$、没学 $p$ 本身。所以 score diffusion 实践里通常不加 MH,靠小 $\delta$ + annealing 控制误差。

5. 与 DDPM 的精确对应(最重要的统一)

到这里你已经看过 Langevin、annealed Langevin 与扩散模型的「类比」性陈述,但「类比」和「相等」之间隔着一道公式。把对应关系写明白:在 DDPM 的反向单步里,把 $\epsilon_\theta(x_t, t)$ 用 Tweedie 恒等式换成 score $s_\theta(x_t, t) = -\epsilon_\theta(x_t, t)/\sqrt{1-\bar\alpha_t}$,再把 $\alpha_t = 1 - \beta_t$ 在小 $\beta_t$ 极限下展开,反向更新可以化简成 $x_{t-1} = x_t + \beta_t\,s_\theta(x_t, t) + \sqrt{\beta_t}\,z$——这正是 noise level $\sigma_t = \sqrt{1-\bar\alpha_t}$ 上的一步 Langevin(步长 $\delta = 2\beta_t$ 的版本)。整条反向链 $T = 1000$ 步就是 1000 个不同 noise level 上各跑一步 Langevin 的串接。

这个对应不只是「数学上恰巧相等」,而是有深刻物理意义:annealed Langevin 是「在固定 $p_\sigma$ 上跑很多步、再切换到更小 $\sigma$」;DDPM 反向是「每个 $\sigma_t$ 只跑一步、立刻切换」。前者更稳但慢,后者更快但每个 level 的样本不够 mixing——所以 DDPM 通常需要数百到上千 timestep 才能补偿。DDIM、DPM-Solver 等加速器本质上是在这条链上做「跳步」与「高阶离散化」,让 20–50 步就能跑完。这也是为什么理解 Langevin 是理解所有 diffusion 加速器的前提。

下方动画里蓝色粒子按 Langevin 在双峰势能面上游走。不变量:粒子轨迹访问 mode A 与 mode B 的时间比例,长时间下来会精确等于两个 mode 的概率比例——即使粒子从不停下,访问频率本身就编码了 $p(x)$。

mode A · 高 $p$ mode B · 高 $p$ 山脊 粒子从高处释放:drift 滚下 + noise 抖动 + 偶尔翻山

动画粒子的运动满足三件事:(1) 大部分时间停在某个山谷底附近——因为 score 在那里指向中心,drift 把它拉回;(2) 即使在山谷底,粒子仍在抖——$\sqrt{\delta}\epsilon$ 这一项永远不为零;(3) 偶尔(概率随 $\delta$ 与山脊高度变化)噪声足够大,粒子被推过山脊到另一侧山谷。这三件事一起,让长时间访问频率严格等于 $p(x)$。换句话说,单条轨迹随时间的遍历平均等于分布的空间平均——这就是遍历定理(ergodic theorem)在 Langevin 上的具体体现,也是为什么我们可以「跑一条长链当一堆样本用」。

手算:2D 高斯混合上跑 Langevin

设目标 $p(x) = 0.5\,\mathcal{N}(x; (-2, 0), I) + 0.5\,\mathcal{N}(x; (2, 0), I)$。两个 mode 在 $x_1 = \pm 2$,方差为 $I$。粒子初始位置 $x_0 = (1, 0)$(在右侧 mode 附近但还没到)。

第 1 步:算 score。因为 $x_0$ 离右侧高斯 $(2, 0)$ 距离为 1,离左侧 $(-2, 0)$ 距离为 3,右侧高斯密度 $\exp(-1/2) \approx 0.61$,左侧 $\exp(-9/2) \approx 0.011$,右侧权重压倒性主导。近似 $\nabla \log p(x_0) \approx \nabla \log \mathcal{N}(x_0; (2,0), I) = (2,0) - (1,0) = (1, 0)$。

第 2 步:代入更新式。取 $\delta = 0.1$,采 $\epsilon_0 = (0.3, -0.2)$:

$$ x_1 = (1, 0) + \frac{0.1}{2}(1, 0) + \sqrt{0.1}(0.3, -0.2) $$

$$ = (1, 0) + (0.05, 0) + (0.0949, -0.0632) = (1.145, -0.063) $$

第 3 步:观察 score 与 noise 的相对量级。drift 贡献 $0.05$,noise 贡献的标准差是 $\sqrt{0.1} \approx 0.316$。noise 比 drift 大 6 倍以上——这不是 bug 而是 feature。每一步看起来主要是随机走,但成千上万步累加下来,drift 的方向偏移会持续指向 mode,而 noise 的方向均匀对消。最终统计意义上 drift 把粒子稳定在 mode 附近、noise 让它探索周围。

第 4 步:跑 1000 步会怎样。粒子会主要在 $(2, 0)$ 附近震荡(占用大约 50% 的时间,因为右侧 mode 权重为 0.5);偶尔(大约每隔几百步)噪声方向恰好向左、累计走过 $x_1 = 0$ 的鞍部,跳到 $(-2, 0)$ 附近,再震荡一段时间。长时间下来两 mode 的访问频率各约 50%——这就是「Langevin 在采样而不是优化」最具体的体现。如果是 SGD,粒子会停在 $(2, 0)$ 永远不动,永远不知道左边还有个 mode。

第 5 步:看 mixing time 对 $\delta$ 的依赖。把 $\delta$ 改成 $0.01$(小 10 倍),单步噪声标准差降到 $0.1$,要跨越 $x_1 = 0$ 鞍部需要累计的随机走步数增加 100 倍——粒子卡在右侧 mode 的时间变得极长,1000 步根本翻不过去;只有跑到 $10^5$ 步以上才会看到 mode 交换。反过来,把 $\delta$ 改成 $1.0$(大 10 倍),噪声标准差变成 $1$,单步位移巨大,粒子频繁跨过鞍部,mode 切换变快,但每个 mode 内部的分布变得偏离 $\mathcal{N}((2,0), I)$(离散化 bias 增大)——样本不再准确反映目标。这就是 $\delta$ 选取的两难:太小不 mixing,太大有 bias。annealed Langevin 用「先大 $\sigma$ 让 modes 糊在一起、再小 $\sigma$ 精修」绕开这个两难。

常见误区

  1. 「Langevin 就是加噪声的 SGD」。反例:SGD 收敛后 $\|\nabla L(x)\| \to 0$、$x$ 停下;Langevin 即便在 mode 中心 $\nabla\log p(x^*) = 0$ 处,$\sqrt{\delta}\epsilon$ 项仍把粒子推开——它永不停下,因为目标是采样不是优化。数值上:在 mode 中心 drift 项贡献 0,单步位移完全由 noise 决定,方差 $\delta\cdot I$,粒子立刻飘走。把 Langevin 跑到「收敛」是误解任务。
  2. 「步长 $\delta$ 越小越准确」。反例:取 $\delta = 10^{-8}$,单步 drift $\sim 10^{-8}$、单步噪声 $\sim 10^{-4}$,1000 步累计噪声 $\sim 10^{-2.5}$——粒子几乎不动,更不可能跨越两个 mode 之间的低概率区。在 2D 双峰例子上,$\delta = 10^{-8}$ 跑 $10^4$ 步、最终位置距离起点 $< 0.1$,只采到初始点局部分布。需要的是合适的 $\delta$ + annealing schedule——一味取小是没有 mixing。这是 Langevin 与优化最反直觉的地方:在 SGD 里小学习率换稳定,在 Langevin 里小步长换不到任何东西,反而把采样变成「停在原地的高斯涨落」,离散 bias 是次要问题,mixing 失败才是首要问题。
  3. 「训完 score 就能直接跑 Langevin 生成图像」。反例:NCSN paper 实验证明,对 CIFAR-10 直接训 score $s_\theta(x) \approx \nabla\log p_{\text{data}}(x)$,然后从 $\mathcal{N}(0, I)$ 出发跑 vanilla Langevin 1000 步——生成结果是纯噪声,FID > 200。原因:图像分布集中在低维流形上,$\mathcal{N}(0, I)$ 采的初始点都在流形外(流形外的体积占绝对多数),score 估计是垃圾、网络从未见过那种输入;同时即便侥幸落在流形附近,CIFAR 的不同类别 mode(飞机/狗/船)之间的低概率区极宽,vanilla Langevin 的噪声完全不够翻越——粒子卡在初始化附近的虚假吸引子里。必须用 annealed Langevin(10+ 个 noise level,$\sigma_1 \approx 50$ 到 $\sigma_L \approx 0.01$)才 work,这是 Song & Ermon 2019 整篇论文的核心实验贡献,也是 score-based diffusion 之所以是「diffusion」而不是「直接采 score」的根本原因。

相关概念

  • Score Function · Langevin 更新式里 drift 项的本体,$\nabla\log p$。
  • Score Matching · 如何在不知道 $Z$ 的情况下学到 $s_\theta \approx \nabla\log p$。
  • SDE Formulation · Langevin 的连续时间母体,以及与 forward/reverse-time SDE 的关系。
  • Probability Flow ODE · Langevin 的确定性同伴,把 SDE 的噪声去掉、共享相同边际。

来源

  1. diffusion-principles-vizuara · L5 · Energy Based Models / Score Function · (00:04)
  2. diffusion-principles-vizuara · L5 · EBM 能量函数与 score 推导 · (09:45)
  3. diffusion-principles-vizuara · L8 · SDE 框架统一 DDPM/score/EBM 三条线 · (00:05)
  4. cmu-10799 · L4 · Score-based Models — Diffusion & Flow Matching · (00:04)
查看 203 条课程原文出处
CourseLectureTimeExcerpt
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 00: 04 So let's begin. Previously last class uh So let's begin. Previously last class uh we have talked about what is diffusion we have talked about what is diffusion we have talked about what is diffusion m…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 00: 57 reserve sorry um all right so just a reserve sorry um all right so just a reminder what is the forward process reminder what is the forward process reminder what is the forward process basically ally …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 01: 44 sums up together is another gausian and you can just sum up a bunch of gausians. you can just sum up a bunch of gausians. you can just sum up a bunch of gausians. So that's why in the forward process …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 02: 35 is the diffusion for process. All right. Now, uh how now that we uh we have the Now, uh how now that we uh we have the Now, uh how now that we uh we have the forward process to train to construct forw…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 03: 24 mean of the gausian uh and in addition we can actually rewrite this mean this we can actually rewrite this mean this we can actually rewrite this mean this prediction of mean into something that predi…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 04: 08 last time there's some questions about why should we do this reparameterization why should we do this reparameterization why should we do this reparameterization well because doing this regression is …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 04: 54 this uh before I do that does anyone know in the audience what is normalizing know in the audience what is normalizing know in the audience what is normalizing flow by any No way. Nobody know. No way.…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 05: 45 to uh latent is the inverse of the mapping from latent to data. So this is mapping from latent to data. So this is mapping from latent to data. So this is sort of like normalizing flows. Um and sort o…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 06: 40 if you have a auto reggressive models if you have a auto reggressive models then you need to break things up by the then you need to break things up by the then you need to break things up by the chai…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 07: 25 And uh for normalizing flow just by looking at this can anyone tell me what looking at this can anyone tell me what looking at this can anyone tell me what potential problem it can it can have. Yes. Y…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 08: 23 least. Um but then it as we have learned last time it also has a problem of like last time it also has a problem of like last time it also has a problem of like very unstable training and also it very…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 09: 24 of images rather than actually having to of images rather than actually having to comput the comput the comput the >> yes that's right so you just need to >> yes that's right so you just need to >> ye…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 10: 13 model so what you do is you sort of just like follow the gra so say suppose you like follow the gra so say suppose you like follow the gra so say suppose you have a gradient that gives you like u have…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 11: 00 space right so basically what you can imagine is that you can start from imagine is that you can start from imagine is that you can start from anywhere on the data space so in this anywhere on the dat…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 11: 45 respect to the data point of the log likelihood so what is happening is that likelihood so what is happening is that likelihood so what is happening is that like so and then again like what you can li…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 12: 31 just like in diffusion right your source distribution and your target distribution and your target distribution and your target distribution should always be in the distribution should always be in th…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 13: 28 gradient of uh of of P X uh with respect to X. That's fine. But X uh with respect to X. That's fine. But X uh with respect to X. That's fine. But later on we're going to see why because later on we're…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 14: 15 is uh machine learning 101 right. So very easy very easy uh uh uh very easy very easy uh uh uh very easy very easy uh uh uh formulation. And this L2 here uh it has formulation. And this L2 here uh it …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 15: 08 like auto reggressive models. So it can like auto reggressive models. So it can sample or generate everything all at sample or generate everything all at sample or generate everything all at once once…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 15: 47 this L2 without this ground truth score. Turns out you don't even need to use the Turns out you don't even need to use the Turns out you don't even need to use the ground truth score in your loss grou…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 16: 35 simplify uh the L2 the the original L2 simplify uh the L2 the the original L2 into two parts here. And then the first into two parts here. And then the first into two parts here. And then the first pa…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 17: 27 can have the gradient of uh log of p with respect to x into you can break it with respect to x into you can break it with respect to x into you can break it into this thing right uh and then these int…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 18: 13 and then you can basically just get something like this. So um yeah this is something like this. So um yeah this is something like this. So um yeah this is a this is like we we probably wouldn't a thi…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 18: 58 need to remember is that the second part of the two things left. It's going to of the two things left. It's going to of the two things left. It's going to come boils down to the negative of the come b…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 19: 41 loss that we get, we don't really need a ground true score. It's it's like ground true score. It's it's like ground true score. It's it's like actually pretty pretty sick. So all you actually pretty p…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 20: 38 function has the same dimensionality of function has the same dimensionality of the of the data, right? Yeah. So, you the of the data, right? Yeah. So, you the of the data, right? Yeah. So, you can ju…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 21: 37 no one else. Do you have a Do you have a Do you have a Do you have a Do you have a >> Well, yes, but it's kind of related to >> Well, yes, but it's kind of related to >> Well, yes, but it's kind of re…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 22: 42 Literally Gausian. Okay. So basically what you do is uh and this is what we what you do is uh and this is what we what you do is uh and this is what we call dnoising score matching. Why so get call dn…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 23: 25 is you just need a sample data and then perturb it a little bit and now it perturb it a little bit and now it perturb it a little bit and now it becomes this like it's easier to becomes this like it's…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 24: 15 literally this. Uh so yeah, for for literally this. Uh so yeah, for for those of you who wonder why log density those of you who wonder why log density those of you who wonder why log density easier b…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 25: 25 truth it's a perturbed ground truth then truth it's a perturbed ground truth then you may have a question but like let's you may have a question but like let's you may have a question but like let's m…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 26: 04 anyway it's fine. Um uh so what you do is you first draw draw a sample from is you first draw draw a sample from is you first draw draw a sample from your data space using some easy to your data space…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 26: 50 So this is what could have happened if you do laundry dynamic essentially. So you do laundry dynamic essentially. So you do laundry dynamic essentially. So at the beginning, right, we just like at the…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 27: 44 anyway you just train your model to predict the score and then you use predict the score and then you use predict the score and then you use dynamic to sample. Uh however does it dynamic to sample. Uh…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 29: 00 survey. Let's survey. Uh we are starting from this column again. from this column again. from this column again. Hello friends. Uh Hello friends. Uh Hello friends. Uh does anyone want to share their t…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 30: 06 haven't been that I haven't been talked haven't been that I haven't been talked to today. There's so many people in the to today. There's so many people in the to today. There's so many people in the …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 31: 12 is what we call the manifold hypothesis. What does it mean is that hypothesis. What does it mean is that hypothesis. What does it mean is that basically usually the data that we want basically usually…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 31: 59 what here says like some area of the what here says like some area of the data space will not have support. What data space will not have support. What data space will not have support. What it means …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 32: 43 manifold from some point from on the manifold to some point that's not on the manifold to some point that's not on the manifold to some point that's not on the manifold the score is going to blow up m…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 33: 28 then the second thing. Yeah. Yeah. Yeah. >> What is this the left plot even showing >> What is this the left plot even showing >> What is this the left plot even showing either? Oh sorry the the the s…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 34: 10 space or entire data space now has support uh and then the score matching support uh and then the score matching support uh and then the score matching actually behave really really nicely actually be…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 34: 57 image is eventually boiled down to say 10 features or something in that case 10 features or something in that case 10 features or something in that case the manifold is load eventually how is the mani…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 35: 45 there's no probability and that's why there's no probability and that's why the score is going to blow up and adding the score is going to blow up and adding the score is going to blow up and adding s…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 36: 42 is like they they they train on the is like they they they train on the clean data. So it's like not I mean yes clean data. So it's like not I mean yes clean data. So it's like not I mean yes obviousl…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 37: 28 without any data right? So what's going to happen here is that um so say this is to happen here is that um so say this is to happen here is that um so say this is like your actual data score that your…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 38: 14 really know where to move in the low density region. So you're just going to density region. So you're just going to density region. So you're just going to move sort of randomly and then you you move…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 39: 02 high density from higher density right high density from higher density right so like say this is your like intended so like say this is your like intended so like say this is your like intended data …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 39: 48 >> it it depends on like what exactly is >> it it depends on like what exactly is this distribution say if this uh this distribution say if this uh this distribution say if this uh distribution is a m…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 40: 34 Okay. All right. Any other question? Cool. Cool. All right. So, we have all these All right. So, we have all these All right. So, we have all these problems. problems. problems. Is it problem? Is it p…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 41: 48 thing doesn't exist anymore, right? And thing doesn't exist anymore, right? And then now the perturbed like say you have then now the perturbed like say you have then now the perturbed like say you ha…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 42: 37 large of a of a noise, then it you're just learning a very noisy distribution. just learning a very noisy distribution. just learning a very noisy distribution. That's not what we want. So how much Th…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 43: 33 about which region that we are uh that about which region that we are uh that we're s we should be sampling at, then we're s we should be sampling at, then we're s we should be sampling at, then we're…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 44: 19 sample from, you are going to just gradually decrease the noise or the gradually decrease the noise or the gradually decrease the noise or the exploration term that you're adding at exploration term t…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 45: 05 have that next? uh we have the pseudo code and you know the long dynam dynamic code and you know the long dynam dynamic code and you know the long dynam dynamic left over. Yeah. But basically literall…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 45: 50 And then at training time you do this multi-level den noising score matching multi-level den noising score matching multi-level den noising score matching because at each time you basically because at…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 47: 23 Yeah, just just think about it. Uh, basically the difference between DDPM, basically the difference between DDPM, basically the difference between DDPM, which is what we learned last last time, which …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 48: 48 that the the the score is sort of it has that the the the score is sort of it has like another factor or it has another like another factor or it has another like another factor or it has another coef…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 49: 49 we have infinite numbers of noise level? we have infinite numbers of noise level? What do you what do we think is going to What do you what do we think is going to What do you what do we think is goin…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 50: 47 tailor estimation of of of the real thing and you can actually write both of thing and you can actually write both of thing and you can actually write both of them into this this format and uh if you …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 51: 42 because you see how like basically if you do xt plus delta t minus xt is going you do xt plus delta t minus xt is going you do xt plus delta t minus xt is going to be this thing right. So it's going t…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 52: 28 you just add some stoasticity. So this you just add some stoasticity. So this is why it's called stochastic is why it's called stochastic is why it's called stochastic differential equation. different…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 53: 28 and we choose the for process right so we know f and g uh and then it's we know f and g uh and then it's we know f and g uh and then it's literally just gausian right so who literally just gausian rig…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 54: 21 to use a oiler solver which is like to use a oiler solver which is like probably simple simplest solver uh that probably simple simplest solver uh that probably simple simplest solver uh that then you…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 55: 09 apply the change in x to x and then we apply the change in x to x and then we apply delta t to t apply delta t to t apply delta t to t and that's it. and that's it. and that's it. And then we iterate …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 56: 13 written as this like this function this written as this like this function this this uh so there are four process can be this uh so there are four process can be this uh so there are four process can …
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 57: 20 the G as variance and in the VE formulation we do not have any formulation we do not have any formulation we do not have any adjustment of the X like whatever X you adjustment of the X like whatever X…
cmu-10799-diffusion-flow L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching 58: 09 did diffusion. This class we did did diffusion. This class we did scorebased model and we also learned scorebased model and we also learned scorebased model and we also learned that they're basically …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 00: 05 Hello everyone and welcome to the next Hello everyone and welcome to the next lecture of the course principles of lecture of the course principles of lecture of the course principles of diffusion mode…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 01: 09 looked at other lectures. This is looked at other lectures. This is supposed to be a standalone lecture to supposed to be a standalone lecture to supposed to be a standalone lecture to help you get in…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 02: 06 are going to discuss today. are going to discuss today. The good thing is that this unified The good thing is that this unified The good thing is that this unified approach has a lot of connection wit…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 03: 13 Okay. So what do we know about diffusion? In this image diffusion? In this image diffusion? In this image I have shown a dye diffusing or I have shown a dye diffusing or I have shown a dye diffusing o…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 04: 37 Can we or in other words can we recover Can we or in other words can we recover the original structure? the original structure? the original structure? So imagine you're sitting and you see So imagine…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 05: 29 shape of the die. But that is not the case. [snorts] But that is not the case. [snorts] But that is not the case. [snorts] So reversing time is something which is So reversing time is something which …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 06: 23 unpredictable. It's it's very random. It's not moving in a specific direction, It's not moving in a specific direction, It's not moving in a specific direction, but but but they're moving up and down.…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 07: 30 So when the dye was diffusing and when So when the dye was diffusing and when we reversed the whole process, we reversed the whole process, we reversed the whole process, can you tell this movement of…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 08: 29 seconds have passed and someone tells me seconds have passed and someone tells me that Rajett this is the particle which that Rajett this is the particle which that Rajett this is the particle which i…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 09: 29 physics and AI comes into the picture. This is a well-known physical property This is a well-known physical property This is a well-known physical property and it makes sense in when you when we and i…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 10: 33 Okay. Now let us apply this concept Okay. Now let us apply this concept to data. to data. to data. Why are we applying this concept to Why are we applying this concept to Why are we applying this conc…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 11: 30 of them have wide eyes. And the question they ask us is, And the question they ask us is, And the question they ask us is, can you find a distribution can you find a distribution can you find a distri…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 12: 35 when you sample from it, you should get when you sample from it, you should get something which lies within that something which lies within that something which lies within that distribution. distrib…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 13: 28 And to find such distribution we need an algorithm which is clever enough which algorithm which is clever enough which algorithm which is clever enough which can understand that okay uh cats have a ca…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 14: 24 the diffusion process. Now this is where it gets very interesting. In the next it gets very interesting. In the next it gets very interesting. In the next set of slides I'm going to replace the set of…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 15: 37 and then later we recover it. So we destroy something only to recover So we destroy something only to recover So we destroy something only to recover it after some point. it after some point. it after…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 16: 36 visualize the dye being slowly filling visualize the dye being slowly filling the liquid and the structure the liquid and the structure the liquid and the structure disappearing disappearing disappear…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 17: 33 do the opposite. In the reverse process, we will start with noise and then slowly we will start with noise and then slowly we will start with noise and then slowly we will move backwards and we will w…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 18: 33 We have already said that we are going We have already said that we are going to apply the diffusion process to data. to apply the diffusion process to data. to apply the diffusion process to data. Bu…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 19: 41 and you see this motion in a lot of physical processes you see these motions physical processes you see these motions physical processes you see these motions in polymers you see this even in the in p…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 20: 36 On the right hand side we are zooming in on specific particles. So here I have on specific particles. So here I have on specific particles. So here I have taken taken taken seven particles as referenc…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 21: 35 So what am I exactly doing here? Well imagine that instead of these tiny dots imagine that instead of these tiny dots imagine that instead of these tiny dots you have 784 dots and all these dots are y…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 22: 35 I'm planning to take all these images, divide them into pixels and subject each divide them into pixels and subject each divide them into pixels and subject each pixel to a brownian trajectory so that…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 23: 43 two terms a drift term and a noise term. two terms a drift term and a noise term. So DDPM has a specific formulation for So DDPM has a specific formulation for So DDPM has a specific formulation for t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 24: 32 Well, the first thing I can see is that I can obviously see some current lines I can obviously see some current lines I can obviously see some current lines in the river. So I'll say that well in the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 25: 24 a steady drift term which is the current of the lake or the river and the second of the lake or the river and the second of the lake or the river and the second term is the diffusion term which are te…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 26: 20 for this forward diffusion. for this forward diffusion. Okay. So we have converted data to noise Okay. So we have converted data to noise Okay. So we have converted data to noise and the change in the…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 27: 18 You might relate this to a variational You might relate this to a variational autoenccoder where there is an encoder autoenccoder where there is an encoder autoenccoder where there is an encoder and t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 28: 09 with our discussion with the time machine. We were asking that can we machine. We were asking that can we machine. We were asking that can we reverse the reverse the reverse the diffusion of the dye? …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 29: 05 more than 40 years back in the year 1982 by a single author Brian Anderson. by a single author Brian Anderson. by a single author Brian Anderson. And in this paper he wrote a formula And in this paper…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 30: 12 appreciate all the researchers who spend appreciate all the researchers who spend a lot of time in writing these papers. a lot of time in writing these papers. a lot of time in writing these papers. O…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 30: 59 of G of T which is the diffusion term in the forward process. And the second term the forward process. And the second term the forward process. And the second term is the score function. is the score …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 31: 52 reversed from noise to the image and reversed from noise to the image and this is where the learning will happen. this is where the learning will happen. this is where the learning will happen. Why is…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 32: 44 where the density of the sharks is maximum. maximum. maximum. Now score function is exactly like this Now score function is exactly like this Now score function is exactly like this compass and sharks…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 33: 44 density. Now where does the gradient and log come Now where does the gradient and log come Now where does the gradient and log come from? Well, the gradient comes from uh from? Well, the gradient come…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 34: 34 to predict. Remember we started with a discussion that we want to predict the discussion that we want to predict the discussion that we want to predict the distribution of the cats and this distributi…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 35: 34 For now let's simplify things. Let us For now let's simplify things. Let us assume that the score function is assume that the score function is assume that the score function is already learned. alrea…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 36: 30 comprises of the drift in the forward comprises of the drift in the forward diffusion and the factor which depends diffusion and the factor which depends diffusion and the factor which depends on the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 37: 26 ask the question what is the variation ask the question what is the variation in the values of these two points. It in the values of these two points. It in the values of these two points. It turns ou…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 38: 15 in terms of delta t. So if you have delta t you calculate delta x then you delta t you calculate delta x then you delta t you calculate delta x then you again substitute delta t calculate delta again …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 39: 04 differential equation or an SDE. This formulation was laid down in a This formulation was laid down in a This formulation was laid down in a paper by Song and Arman in the year paper by Song and Arman…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 40: 02 represents the noise term. So we go from the first iteration to the second the first iteration to the second the first iteration to the second iteration. Then we again do diffusion iteration. Then we …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 40: 54 you because we know the exact equation of the reverse process and we use that of the reverse process and we use that of the reverse process and we use that same equation for sampling. same equation fo…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 41: 49 data set consisting only of images of data set consisting only of images of handwritten digits which are ones and handwritten digits which are ones and handwritten digits which are ones and eights. ei…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 42: 46 like these handwritten digits. Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal Okay. So uh we are expecting a biodal distribution. This is what where we want distribution. …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 43: 41 in the image follows a brownie in motion. motion. Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing Okay. So just like a paint is diffusing in water our in water our in w…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 44: 40 trajectory I have shown but in reality we have these 784 pixels. So it will be we have these 784 pixels. So it will be we have these 784 pixels. So it will be in a 784dimensional space which is in a 7…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 45: 31 Now the next step is we know that the microscopic brownian motion is microscopic brownian motion is microscopic brownian motion is reversible. Uh because as you can see we reversible. Uh because as yo…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 46: 28 as well. And now you can see on the left hand And now you can see on the left hand And now you can see on the left hand side we have noise and we are using the side we have noise and we are using the …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 47: 16 diffusion term but this time the trajectory is spread into two very trajectory is spread into two very trajectory is spread into two very nicely and uh we do get a biodal nicely and uh we do get a bio…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 48: 08 have seen this word score function in many papers but only after going through many papers but only after going through many papers but only after going through the SD framework we really understand t…
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 48: 59 using a very simple approach which can using a very simple approach which can be implemented in Google Collab and uh be implemented in Google Collab and uh be implemented in Google Collab and uh this …
diffusion-principles-vizuara L3: Lecture 8 - Diffusion Models: A Physical Intuition (SDE Framework) | Principles of Diffusion Models 49: 50 It's a very nicely written paper and after this introduction you should be after this introduction you should be after this introduction you should be able to understand it. Thank you very able to und…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 00: 04 Hello everyone and welcome to this next Hello everyone and welcome to this next lecture in of the course principles of lecture in of the course principles of lecture in of the course principles of dif…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 01: 15 of X. And our objective then becomes to And our objective then becomes to And our objective then becomes to determine determine determine the probability distribution P5 of X the probability distribut…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 02: 30 The objective is to match the output image with the input image as close as image with the input image as close as image with the input image as close as possible. possible. possible. And here we look…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 03: 46 intelligence can be used to produce intelligence can be used to produce images which look like they belong to images which look like they belong to images which look like they belong to the underlying…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 04: 42 it becomes uniform. We took these key ideas and we We took these key ideas and we We took these key ideas and we constructed a diffusion model which is constructed a diffusion model which is construct…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 05: 49 So you inject noise at every transition. So this is the first transition, second, So this is the first transition, second, So this is the first transition, second, third etc. third etc. third etc. Now…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 06: 45 So our objective is to minimize the mean So our objective is to minimize the mean square error between epsilon and epsilon square error between epsilon and epsilon square error between epsilon and eps…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 07: 41 something which is common in every single image on earth. And at a first single image on earth. And at a first single image on earth. And at a first glance you're not able to answer it glance you're n…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 08: 36 in the field of image generation through generative AI. Models were extremely generative AI. Models were extremely generative AI. Models were extremely important in creating a new wave in the importan…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 09: 45 models in a single cohesive framework. models in a single cohesive framework. So this is something which is very So this is something which is very So this is something which is very unique and and ve…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 10: 48 So in the energy based models our So in the energy based models our objective is to predict P5 of X using objective is to predict P5 of X using objective is to predict P5 of X using energy functions. …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 11: 57 drop this ball in this curve. What will happen is that the ball will What will happen is that the ball will What will happen is that the ball will slowly settle down, slowly fall down and slowly settl…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 12: 57 down and settle on the ground? The reason is that at on on on the ground reason is that at on on on the ground reason is that at on on on the ground the potential energy of the apple is is the potenti…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 13: 55 energy function can be a proxy for how energy function can be a proxy for how probable that particular data point or probable that particular data point or probable that particular data point or sampl…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 14: 52 course draw this schematically and we can say that okay we want this energy or can say that okay we want this energy or can say that okay we want this energy or or this probability curve to follow or …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 15: 51 my function should satisfy my function should satisfy and people use an exponential function and people use an exponential function and people use an exponential function to relate the energy to the p…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 17: 03 density as p5 of x. density as p5 of x. Then from the above graph we can find Then from the above graph we can find Then from the above graph we can find out a function which represents the out a func…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 17: 56 system u every system finally tries to go in a position with the minimum go in a position with the minimum go in a position with the minimum possible energy and this is something possible energy and t…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 18: 46 the whole idea behind energy based models models okay now let us discuss after this point okay now let us discuss after this point okay now let us discuss after this point let's take an example suppos…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 20: 00 add up to one? We need the probability to add up to one because if we are to add up to one because if we are to add up to one because if we are creating a lot of samples and we are creating a lot of s…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 20: 52 so let's let's take that example again. so let's let's take that example again. Uh let's say you have a dice. Okay. Uh Uh let's say you have a dice. Okay. Uh Uh let's say you have a dice. Okay. Uh and…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 21: 42 and the solution is that we can simply and the solution is that we can simply normalize the probabilities by dividing normalize the probabilities by dividing normalize the probabilities by dividing it…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 22: 48 So for the discrete states it looked So for the discrete states it looked like this summation of e ra to e5 of x like this summation of e ra to e5 of x like this summation of e ra to e5 of x x goes fr…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 23: 46 energy based models. In other words, how do we find these energy functions which do we find these energy functions which do we find these energy functions which correspond to each of these states? cor…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 24: 58 push the good data up. push the good data up. So I want to move this curve like this. So I want to move this curve like this. So I want to move this curve like this. The left side I want to push down …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 25: 49 higher probability. All of us understand higher probability. All of us understand this. this. this. Now we will again use the same Now we will again use the same Now we will again use the same formula…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 26: 47 Uh here E5 of X denotes the energy function. Okay. So now if you break this down, Okay. So now if you break this down, this is the first term that you get. uh this is the first term that you get. uh t…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 28: 00 so p5 of x can be written as integral of p5 of x given z into p of z dz. p5 of x given z into p of z dz. p5 of x given z into p of z dz. And there we said that this integral is And there we said that …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 29: 00 bypasses the partition function. So you might be thinking that what is So you might be thinking that what is So you might be thinking that what is this new terminology which I've this new terminology …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 29: 56 And because of that we come up with a alternative objective function which alternative objective function which alternative objective function which uses something called as the score uses something c…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 31: 05 appreciation for people who actually sit appreciation for people who actually sit down, they do the mathematics and they down, they do the mathematics and they down, they do the mathematics and they c…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 32: 17 to occur. to occur. Let's let let's take a sample example. Let's let let's take a sample example. Let's let let's take a sample example. Let us say these black dots these are Let us say these black do…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 33: 18 score function does. It tells you the score function does. It tells you the direction to where the data is located. direction to where the data is located. direction to where the data is located. So i…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 34: 09 know where the data is where the score is the maximum is the maximum is the maximum we can start from any point and finally we can start from any point and finally we can start from any point and fina…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 34: 58 trajectory? Is is is it can I define it? Can I write Is is is it can I define it? Can I write Is is is it can I define it? Can I write an equation which can tell me how do I an equation which can tell…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 35: 58 Let's say I calculate the score for this Let's say I calculate the score for this point. Where will this score point point. Where will this score point point. Where will this score point towards to th…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 36: 53 right or the left uh the value will decline at x= to infinity or minus decline at x= to infinity or minus decline at x= to infinity or minus infinity the value becomes zero. infinity the value becomes…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 37: 55 Now this is just the first term. We have to take a gradient of this also. to take a gradient of this also. to take a gradient of this also. Gradient of a constant is zero and the Gradient of a constan…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 38: 44 So uh this is what we get if we actually plot the score function. You can see plot the score function. You can see plot the score function. You can see that the arrows are pointed towards the that the…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 39: 41 score-based framework but score-based framework but score-based methods score-based methods score-based methods is something which has come before is something which has come before is something which…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 40: 40 challenges which we started out with. We wanted something which can uh help us wanted something which can uh help us wanted something which can uh help us offer a tractable alternative to offer a trac…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 42: 09 Now what what happens is that when you take this gradient uh the the first term take this gradient uh the the first term take this gradient uh the the first term is minus grad of e5 of x which is simp…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 43: 06 that's all. So the partition function becomes zero So the partition function becomes zero So the partition function becomes zero and is not included in the calculation and is not included in the calcu…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 44: 02 makes sure that the score of the predicted model matches the score of the predicted model matches the score of the predicted model matches the score of the true probability distribution. true probabil…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 45: 33 So ideally we want a loss which looks like this. Now the a loss which looks like this. Now the a loss which looks like this. Now the question again becomes how do we question again becomes how do we q…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 46: 22 Let us assume that we have done the training. We have matched this score training. We have matched this score training. We have matched this score with the uh score for the true model for with the uh …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 47: 09 data point and I know where to go I can navigate my way and I can find samples navigate my way and I can find samples navigate my way and I can find samples from the distribution Okay. And uh this fro…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 48: 18 Here I can visually see that this is the deepest valley and I can see the deepest valley and I can see the deepest valley and I can see the treasure is hidden there. treasure is hidden there. treasure…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 49: 10 valley and I start here. and I want to reach the bottom of the valley. So I'll reach the bottom of the valley. So I'll reach the bottom of the valley. So I'll make a step like this. I'll again make a …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 50: 12 would get your next location. But what is the slope here? How do we But what is the slope here? How do we But what is the slope here? How do we find the slope? This is my energy find the slope? This i…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 51: 07 exactly what we are doing here. We are exactly what we are doing here. We are calculating the gradient of the energy calculating the gradient of the energy calculating the gradient of the energy and u…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 51: 59 this which is gradient of the energy function. So the negative of this function. So the negative of this function. So the negative of this actually is the score function itself. actually is the score …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 52: 52 gradient is zero. So you are just sitting there but there is another pit sitting there but there is another pit sitting there but there is another pit which is even below which is even below which is …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 53: 40 need to provide a shake which gives you enough random energy to kick you out of enough random energy to kick you out of enough random energy to kick you out of these small potholes so that you can the…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 54: 29 we are sampling from a gshian we are sampling from a gshian distribution with a mean of xi and a distribution with a mean of xi and a distribution with a mean of xi and a standard deviation of beta. s…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 55: 30 the total variance of this xt + 1 is maybe a constant or it's just one that maybe a constant or it's just one that maybe a constant or it's just one that is my hypothesis is my hypothesis is my hypoth…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 56: 32 Now we what we do is we replace this Now we what we do is we replace this grad of energy by s of x which we have grad of energy by s of x which we have grad of energy by s of x which we have already s…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 57: 16 trajectory let's say I start from here with this update I'll go like this with this update I'll go like this with this update I'll go like this and then I'll finally I I I need to come and then I'll f…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 57: 59 the score function which takes you from one location to the another and finally one location to the another and finally one location to the another and finally you end up at a space or a region where …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 58: 55 x is known to us. It looks something like this. The probability of finding like this. The probability of finding like this. The probability of finding the samples is maximum at these two the samples i…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 59: 55 comments. So first we define the setup which are So first we define the setup which are So first we define the setup which are the two mountain setup as as we saw the two mountain setup as as we saw t…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 60: 40 noise. We use the exact same equation which we discuss in theory which we discuss in theory which we discuss in theory and then we visualize the path. and then we visualize the path. and then we visua…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 61: 28 It's it's called a drunk hiker. It's like someone is drunk and walking It's like someone is drunk and walking It's like someone is drunk and walking here and there and finally reaching the here and th…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 62: 18 So I want all of you to run this tweak a little things here and there have fun little things here and there have fun little things here and there have fun with the notebook and really develop with the…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 63: 15 since sampling with langu dynamics only requires the score like we saw in this requires the score like we saw in this requires the score like we saw in this formula it only requires the score we formu…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 64: 27 and score matching fits this vector and score matching fits this vector field by minimizing the mean square field by minimizing the mean square field by minimizing the mean square error between the tr…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 65: 40 know how to pronounce this name. uh however they they came up with a uh however they they came up with a uh however they they came up with a technique to technique to technique to distill that objecti…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 66: 47 a summation of this and a constant. So basically minimizing this meant that So basically minimizing this meant that So basically minimizing this meant that we want to minimize this and you see here we…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 68: 01 like this. A1 A2 A3 A4 A5 A6 A4 A5 A6 A4 A5 A6 A7 A8 and A9. So the trace of this A7 A8 and A9. So the trace of this A7 A8 and A9. So the trace of this matrix is simply the addition of a1 + a5 matrix …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 69: 11 uh it is it is completely fine for our uh it is it is completely fine for our purposes what we need to understand is purposes what we need to understand is purposes what we need to understand is the i…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 70: 05 Which makes a lot of sense, right? In Which makes a lot of sense, right? In the previous example that we looked at the previous example that we looked at the previous example that we looked at these t…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 71: 14 where we are trying to make the data samples appear as syncs. samples appear as syncs. samples appear as syncs. which is what you see over here also. If which is what you see over here also. If which …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 72: 07 more to this term. So the this term drives So the this term drives So the this term drives scores to zero in the higher probability scores to zero in the higher probability scores to zero in the highe…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 73: 01 Now uh once we have understood this, Now uh once we have understood this, this is where the actual training of the this is where the actual training of the this is where the actual training of the uh …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 73: 57 distribution. So in a way we will take a look at the whole uh whole pipeline. look at the whole uh whole pipeline. look at the whole uh whole pipeline. Okay, let's try to understand this. Okay, let's …
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 75: 18 loss is made up of two terms. The first loss is made up of two terms. The first is the trace of the Jacobian is the trace of the Jacobian is the trace of the Jacobian which is what we discussed before…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 76: 10 predictions for the x score and the y score to create a vector. score to create a vector. score to create a vector. Next we have a training loop which is a Next we have a training loop which is a Next…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 77: 00 completely appear like a sync but there completely appear like a sync but there are arrows coming in from the left and are arrows coming in from the left and are arrows coming in from the left and the…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 77: 49 we go to any other point. So we are we go to any other point. So we are going to see trajectories like these. going to see trajectories like these. going to see trajectories like these. And in this ex…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 78: 33 perturve ourselves from the valleys so perturve ourselves from the valleys so that we don't miss any other treasures that we don't miss any other treasures that we don't miss any other treasures which…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 79: 19 them yet. We will make the connection in the next lecture. the next lecture. the next lecture. So initially used to enable efficient So initially used to enable efficient So initially used to enable e…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 80: 15 Scores are given by gradient of logarithm of uh the probability. It logarithm of uh the probability. It logarithm of uh the probability. It turns out that the scores appear as turns out that the score…
diffusion-principles-vizuara L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models 81: 02 neural network to match the scores and neural network to match the scores and to do that we use a score matching loss to do that we use a score matching loss to do that we use a score matching loss wh…