Forward Diffusion Process
— 零参数的 Gaussian Markov 链,把任意数据漂洗成 $\mathcal{N}(0,I)$ 噪声。
很多人第一次接触扩散模型时,会把 forward process 想成"一个需要训练的 encoder"——毕竟在 VAE 里 encoder 是要训练的,它学的就是 $q(z \mid x)$ 的均值和方差。如果你刚从 VAE 章节走过来,这种"前向 = encoder"的类比几乎是本能。这是要立刻纠正的偏见:扩散里的 forward 不是一个网络,它是纯解析的——一条事先写好系数的 Gaussian 加噪链,没有任何可学习参数、没有梯度回传、训练前训练后完全一模一样。换句话说,VAE 里 $q_\phi(z \mid x)$ 那个下标 $\phi$ 是网络参数;diffusion 里 $q(x_t \mid x_{t-1})$ 没有下标,因为没有参数可学。Vizuara 的 Dr. Raj 在 L3 用一个特别朴实的方式把它讲清楚:讲师本人就是那台"加噪机器"——他在白板上写下规则,你按这条规则把图像反复"漂洗",不需要"教"它怎么加噪,加噪规则本身就是答案。
训练里前向只出现在一个地方:给定一个干净样本 $x_0$、随机采一个 timestep $t \in \{1, \dots, T\}$、再随机采一个 $\epsilon \sim \mathcal{N}(0, I)$,用闭式跳步公式一次合成 $x_t$。不需要真的逐步跑 1000 步——为什么不需要、为什么能跳,这是后面 deepdive 的核心。CMU 10799 的 lecturer 在 L2 引入 DDPM 时把第一句话留给整门课的主旋律:
把它定位到机器学习的上下文里:在训练里,forward 是批量生成训练数据 $x_t$ 的工具——它把"一堆无标注的图像 $\{x_0\}$"转成"成对 $(x_t, t, \epsilon)$",让 reverse 那个真正的网络 $\epsilon_\theta$ 有 supervision 可学。前向相当于一个"自动数据合成器",免费、解析、可并行、和模型权重完全解耦。这一点想清楚后,DDPM 的 algorithm 1 那五行伪代码——为什么前向只在第 4 行用一次乘加就完事——你就理解了它能这样写的真正原因。
单步加噪:定义看公式就够
给定上一步的 $x_{t-1}$,下一步的分布是:
$$q(x_t \mid x_{t-1}) = \mathcal{N}\!\big(x_t;\ \sqrt{1-\beta_t}\,x_{t-1},\ \beta_t I\big)$$
这是一条 Markov 链——下一步 $x_t$ 只依赖当前 $x_{t-1}$,跟更早的 $x_{t-2}, x_{t-3}, \dots$ 无关。用 reparameterisation trick 把它从"分布"写成"可采样的等式":$x_t = \sqrt{1-\beta_t}\,x_{t-1} + \sqrt{\beta_t}\,\epsilon_t,\ \epsilon_t \sim \mathcal{N}(0, I)$。两种写法等价——一个是概率密度形式,一个是采样形式。
Why $\sqrt{1-\beta_t}$ 这个看上去奇怪的系数? 它的作用是把 $x_{t-1}$ 按比例缩小一点点。如果省掉它(只写 $x_t = x_{t-1} + \sqrt{\beta_t}\,\epsilon$,纯粹"叠加噪声"),那么每一步方差都会单调累加,T 步后总方差是 $\sum_s \beta_s$——如果 schedule 累加超过 1,$x_T$ 的方差就会爆炸到 $\mathcal{N}(0, \infty)$,远远偏离我们想要的 $\mathcal{N}(0, I)$ 终点先验。两个系数 $\sqrt{1-\beta_t}$ 与 $\sqrt{\beta_t}$ 平方和恰为 1,是精心选的variance-preserving 形式(VP-SDE 命名就来自这里):每一步同时"把信号按 $\sqrt{1-\beta_t}$ 比例衰减"+"按 $\sqrt{\beta_t}$ 比例注入新噪声",把 $x_t$ 的总方差稳定在 1 附近——只要 $x_0$ 也归一化到方差 1(DDPM 论文里图像被映射到 $[-1, 1]$ 区间),整条链上每个 $x_t$ 的方差都接近 1。这条约束让深网络看到的输入尺度始终稳定,是训练数值上能跑得起来的关键前提。
闭式跳步:把 t 步 unroll 成一次乘加
朴素的做法是想生成 $x_{873}$ 就老老实实从 $x_0$ 跑 873 步采样——每步采一个新 $\epsilon_s$、做一次乘加。这显然太贵:训练里我们每个 batch 都要随机挑一个 $t$,最坏情况下平均要算 $T/2 \approx 500$ 步 forward,对 batch size 256、$T = 1000$ 的标准设定就是 12.8 万次乘加 / batch。如果非这么算,DDPM 训练在 2020 年根本跑不起来。
关键的代数事实把整件事拯救出来:两个独立 Gaussian 相加还是 Gaussian,均值相加、方差相加。把这一性质递归地应用到 forward 链上:
$$x_1 = \sqrt{1-\beta_1}\,x_0 + \sqrt{\beta_1}\,\epsilon_1$$
$$x_2 = \sqrt{1-\beta_2}\,x_1 + \sqrt{\beta_2}\,\epsilon_2 = \sqrt{(1-\beta_2)(1-\beta_1)}\,x_0 + \underbrace{\sqrt{(1-\beta_2)\beta_1}\,\epsilon_1 + \sqrt{\beta_2}\,\epsilon_2}_{\text{两个独立 Gaussian 的和}}$$
右边方括号那一坨——两个独立 Gaussian 的线性组合——的分布完全由它的均值和方差决定:均值为 0,方差是 $(1-\beta_2)\beta_1 + \beta_2$。化简一下:$(1-\beta_2)\beta_1 + \beta_2 = \beta_1 - \beta_1\beta_2 + \beta_2 = 1 - (1 - \beta_1 - \beta_2 + \beta_1\beta_2) = 1 - (1-\beta_2)(1-\beta_1)$。所以可以把那一坨噪声合并成一个新的 Gaussian $\sqrt{1-(1-\beta_1)(1-\beta_2)}\,\bar\epsilon$,其中 $\bar\epsilon \sim \mathcal{N}(0, I)$。把这个递归推到 $t$ 步,引入 $\alpha_s = 1 - \beta_s$ 和累积积 $\bar\alpha_t = \prod_{s=1}^{t}\alpha_s$,就得到 DDPM 的招牌公式:
$$\boxed{\,x_t = \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1-\bar\alpha_t}\,\epsilon,\quad \epsilon \sim \mathcal{N}(0, I)\,}$$
Why 这是 diffusion 训练能 tractable 的全部原因:训练 loop 里我们要随机采 $t \in \{1,\dots,T\}$,再生成对应的 $x_t$。如果没有闭式跳步,采到 $t = 873$ 就要先跑 873 步前向、各采各的 $\epsilon$,一个 batch 一次 forward 就是 $O(T)$ 复杂度;考虑到 DDPM 标准配置 $T = 1000$,根本不可能。有了上式,无论 $t$ 多大,合成 $x_t$ 都只需要一次标量乘加——计算复杂度从 $O(T)$ 降到 $O(1)$,与时间步无关。Ho et al. 2020 把这一公式写在 algorithm 1 的第 4 行(紧接着第 5 行的 loss 计算):整个 DDPM 训练才得以在单张消费级 GPU 上跑得动。这个看似平淡的"高斯加高斯还是高斯"的代数事实,是 diffusion 能成为 scalable 生成模型族的分水岭——所有 SDE、flow-matching、rectified-flow、consistency model 后续工作都建立在"前向可解析跳步"这个前提上。
$T \to \infty$ 的端点:信息被洗净,所有起点合并
当 $t$ 足够大、且 schedule 累积下去使 $\bar\alpha_t \to 0$:
$$x_t = \underbrace{\sqrt{\bar\alpha_t}}_{\to 0}\,x_0 + \underbrace{\sqrt{1-\bar\alpha_t}}_{\to 1}\,\epsilon \ \longrightarrow\ \mathcal{N}(0, I)$$
这个极限有两个等价但角度不同的解读,都值得停下来想一想。几何解读:$x_0$ 那一项的系数 $\sqrt{\bar\alpha_t}$ 单调下降到 0,意味着原始数据被"按比例压缩到原点";同时 $\epsilon$ 那一项系数升到 1,把标准正态噪声完整覆盖进来。最后 $x_T$ 看不出任何 $x_0$ 的痕迹。信息论解读:互信息 $I(x_T; x_0) \to 0$——给你 $x_T$,你猜不出 $x_0$ 是什么。无论起点 $x_0$ 是 Batman 海报、还是一只熊猫、还是手写数字 5,终点都是同一个 $\mathcal{N}(0, I)$。Vizuara 在 L3 里把这一点说得很形象:
这个性质不是 happy coincidence——它正是 forward 被设计成 variance-preserving + 累积 $\bar\alpha_T \approx 0$ 的目的。推理时为什么能从 $\mathcal{N}(0, I)$ 直接起步? 因为前向已经"承诺"了所有数据点都被映射到这同一个先验。如果终点不是已知分布,推理时就根本不知道从哪儿采 $x_T$——这一约束是整个 diffusion 范式能反向生成的前提。也是 forward 这条看似"无用"的破坏过程为什么如此重要的原因:它不是为了破坏,是为了建立一座从数据到先验的、可解析、可逆向的桥。后面 reverse 网络要做的,就是学着沿这座桥反向走。
图示:前向链与 SNR 曲线
把上面的代数翻译成视觉:左侧是干净图,右侧是纯噪声,中间是不同 $t$ 的"半噪声"。同时画出信噪比 $\mathrm{SNR}(t) = \bar\alpha_t / (1-\bar\alpha_t)$——它把"信号"和"噪声"的比例直接量化出来,是同一个前向过程从两个互补角度(图像可视化 vs 数值曲线)的呈现。
上方:图像沿 forward 链颜色渐淡(剩余信号 $\sqrt{\bar\alpha_t}$ 比例下降,噪声点逐帧增多);下方:信噪比单调下降,从 $+\infty$ 跌到 $\approx 0$。noise schedule 控制的就是这条 SNR 曲线的形状——线性 schedule 是凸的、cosine 是 S 形、sigmoid 是中段平台型,各有不同退化节奏。Cosine schedule (Improved DDPM, Nichol & Dhariwal 2021) 之所以效果好,是因为它让中间步停留得更久、给训练更多"中等噪声水平"的样本——这些 $t$ 才是 reverse 网络最有效的学习范围。
逐像素视角:一张 28×28 图怎么被加噪
Dr. Raj 在 L3 给了一个特别具体的例子:拿 MNIST 28×28 = 784 个像素的图,看第一个像素 $x_0^{(1)} = 0.5$。一步加噪相当于对这 784 个像素分别做一次独立 Gaussian 采样:
$$x_1^{(i)} \sim \mathcal{N}\!\big(\sqrt{1-\beta_1}\cdot x_0^{(i)},\ \beta_1\big),\quad i = 1, 2, \dots, 784$$
每个像素的均值是它自己缩水的版本,方差是 $\beta_1$。各像素的噪声 $\epsilon^{(i)}$ 独立同分布——这就是为什么前向噪声叫"i.i.d. 各向同性"。如果用矩阵记号,写成 $x_1 = \sqrt{1-\beta_1}\,x_0 + \sqrt{\beta_1}\,\epsilon$,$\epsilon \in \mathbb{R}^{784}$ 是一个 784 维标准正态向量。整图被 forward 推一步——计算上就是一次逐元素乘法 + 一次逐元素加,向量化在 GPU 上是 $O(d)$,$d$ 是像素数。对 256×256 RGB 图($d \approx 2 \times 10^5$),加噪比一次卷积便宜上百倍——前向几乎不消耗算力。这就是 dataloader 里直接在 CPU 用 numpy 跑前向都不卡的原因。
🟡 教材外补充:从离散链到连续 SDE,从 VP 到 VE 与 rectified-flow
讲师课堂上讲的是离散 $T = 1000$ 的 DDPM 版本——这是 2020 年的写法。Song & Ermon 2021("Score-Based Generative Modeling through Stochastic Differential Equations")把 $T \to \infty$ 的极限写成连续 SDE:$\mathrm{d}x = -\tfrac{1}{2}\beta(t)\,x\,\mathrm{d}t + \sqrt{\beta(t)}\,\mathrm{d}w$,即 variance-preserving SDE(VP-SDE)。在这一连续视角下,"$\sqrt{1-\beta_t}$ 系数"等价于漂移项 $-\tfrac{1}{2}\beta(t)\,x$ 让信号按指数衰减,"$\sqrt{\beta_t}$ 注入的噪声"等价于扩散项 $\sqrt{\beta(t)}\,\mathrm{d}w$ 注入 Brownian motion 增量。课程里学到的"$\bar\alpha_t = \prod(1-\beta_s)$"这个累积积,在连续极限下就变成积分 $\bar\alpha(t) = \exp\!\big(-\int_0^t \beta(s)\,\mathrm{d}s\big)$——离散的乘积变成连续的指数衰减。两种视角等价,离散更适合写代码,连续更适合做数学分析(如证明可逆、推出 score-matching 等价、构造 probability-flow ODE)。
EDM (Karras 2022) 进一步指出:把这同一个 forward 用variance-exploding 形式重新参数化 $x_t = x_0 + \sigma_t\,\epsilon$(不缩信号、只单向加噪),数值上比 VP 更稳——避免了 $\sqrt{\bar\alpha_t}$ 在 $t$ 大时接近 0 带来的数值病态。SD3 / Flux 等 2024 模型默认走 rectified-flow / VE 路线,已经不再写成 $\beta_t$ 链。Rectified flow(Liu et al. 2022)更激进,直接用线性插值 $x_t = (1-t)\,x_0 + t\,\epsilon$ 作为前向——把 noise schedule 设计退化成一条直线。但所有这些 forward 的本质共同点没变:解析、零参数、有闭式跳步。这条主线贯穿过去五年所有 diffusion / flow 文章——理解了 DDPM 的前向,再看任何变体都只是"换不同的 schedule 系数 / 改写参数化"。
点击按钮在四个时刻间切换;右侧数字直接由跳步公式算出。不变量:无论选哪个 $t$,$x_t$ 的均值与 $x_0$ 同方向、按 $\sqrt{\bar\alpha_t}$ 比例缩;剩余方差恰好 $1-\bar\alpha_t$——两者平方和为 1(variance preserving)。
手算跳步:从 $x_0$ 到 $x_3$,并验证与逐步链同分布
用一个极小的 schedule 做手算:$T = 3$,$\beta_1 = 0.1,\ \beta_2 = 0.2,\ \beta_3 = 0.3$(实际 DDPM 的 $\beta$ 都很小,这里用大数字方便用笔算清楚)。先把 $\alpha_s = 1-\beta_s$ 算出来:$\alpha_1 = 0.9,\ \alpha_2 = 0.8,\ \alpha_3 = 0.7$。
累积积(这是 schedule 阶段一次性算完的查表项):$\bar\alpha_1 = 0.9,\ \bar\alpha_2 = 0.9 \cdot 0.8 = 0.72,\ \bar\alpha_3 = 0.72 \cdot 0.7 = 0.504$。
取一个简单的训练样本 $x_0 = 4$,假设标准 Gaussian 采到 $\epsilon = 1$。一次乘加跳到 $t = 3$:
$$x_3 = \sqrt{\bar\alpha_3}\,x_0 + \sqrt{1-\bar\alpha_3}\,\epsilon = \sqrt{0.504}\cdot 4 + \sqrt{0.496}\cdot 1 = 0.710 \cdot 4 + 0.704 \cdot 1 = 2.840 + 0.704 = 3.544$$
验算 Markov 链:从 $x_0 = 4$ 一步步采(这次假设三次都恰好采到 $\epsilon = 1$,是为了让你看清整条链的结构;现实中三个 $\epsilon$ 独立随机):
$x_1 = \sqrt{0.9}\cdot 4 + \sqrt{0.1} \cdot 1 = 3.795 + 0.316 = 4.111$;$x_2 = \sqrt{0.8} \cdot 4.111 + \sqrt{0.2} \cdot 1 = 3.677 + 0.447 = 4.124$;$x_3 = \sqrt{0.7}\cdot 4.124 + \sqrt{0.3}\cdot 1 = 3.451 + 0.548 = 3.999$。
逐步链得到 $x_3 \approx 4.00$,跳步公式得到 $x_3 \approx 3.54$。具体数值不一样是正常的、也是必须的——因为两条路径采到的 $\epsilon$ 序列不同,单条 sample path 当然不一样。但两种采法的 $x_3$ 边缘分布完全相同:均值都是 $\sqrt{0.504}\cdot 4 \approx 2.840$、方差都是 $1 - 0.504 = 0.496$。这正是闭式跳步的全部魔力——跳过中间所有采样、直接采终点的边缘分布,跳过的是 path 不是 marginal。训练 loss 只关心 marginal(loss 里只看 $x_t$ 不看 $x_{t-1}$),所以跳过 path 完全不影响梯度,这也是为什么 DDIM 后来能用同一个权重换 path 的根源。
常见误区
- "$\beta_t$ 是 $x_t$ 时刻的总噪声方差"。反例:$\beta_t$ 是这一步新增的方差,不是累积。累积方差是 $1-\bar\alpha_t = 1 - \prod_{s=1}^{t}(1-\beta_s)$。常见 DDPM 取 $\beta_t \in [10^{-4}, 0.02]$ 线性插值,单步看每个 $\beta_t$ 都很小(最大 0.02 = 2%);但 $T = 1000$ 时累积下来 $\bar\alpha_T \approx e^{-\sum \beta_t} \approx e^{-10} \approx 5 \times 10^{-5}$,所以 $1 - \bar\alpha_T \approx 1$——每步加一点点,累积加完整。理解这一点对调 schedule 至关重要:如果只看 $\beta_t$ 数值小就以为噪声"加得不多",会完全错估终点的退化程度。
- "扩散步数 $T$ 越大效果越好,越接近真实 SDE"。反例:EDM (Karras 2022) 用 $T = 18$ + 二阶 Heun solver,FID 比 DDPM 标准 $T = 1000$ 的设置还低。原因是:$T$ 大本质上是"用更细的步长摊薄一阶 Euler 离散化误差"的笨办法;用二阶/三阶 ODE solver 可以在更少步内得到同等甚至更高精度,因为高阶 solver 每步的局部误差是 $O(\Delta t^k)$($k$ 是阶数),比一阶的 $O(\Delta t)$ 衰减快得多。更好的 solver + 更聪明的 schedule(如 EDM 的 $\sigma$-spaced schedule)直接打破"$T$ 越大越好"这条朴素经验法则。这一误区在 2022 年后的工业部署里影响很大——Stable Diffusion 切到 DPM-Solver++ 后步数从 50 降到 20,速度翻倍而质量不掉。
- "前向加噪 = 把图模糊"。反例:前向是逐像素 i.i.d.加 Gaussian 白噪声,频谱是平坦的——所有空间频率被同等抬高;模糊(低通滤波 / Gaussian blur)只会保留低频、抹掉高频。如果你对 forward 链的中间帧 $x_{500}$ 画 power spectrum,会看到一条平坦的曲线叠加在 $x_0$ 的原始频谱上方;如果是模糊操作,会看到高频段被压扁、变成低通形状。两者数学上完全不同:噪声引入的是新的、独立的、各向同性的方差;模糊只是把已有信号在频域重排,没有引入任何新的随机性。这正是 cold diffusion (Bansal et al. 2022) 研究的动机——他们把 Gaussian 加噪替换成 blur、masking、down-sampling 等确定性退化,发现训练目标、loss 推导、reverse 写法全都要重做,不再是 DDPM 那套理论——足以说明 forward 用 Gaussian 不是"随便选了一种破坏方式"。
相关概念
- DDPM(前向是 DDPM 训练 pipeline 的"数据合成器"——闭式跳步公式直接出现在 Ho 2020 algorithm 1 的第 4 行,整个训练循环就建立在这一行能 $O(1)$ 完成的前提上)
- Reverse Diffusion Process(前向负责"破坏",reverse 负责"还原"——前向的解析性恰恰是反向必须用神经网络估算的根源:每步采的 $\epsilon$ 在前向被丢弃了,反向只能用模型估其期望)
来源
查看 193 条课程原文出处
| Course | Lecture | Time | Excerpt |
|---|---|---|---|
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 00:
03 | Cool. Hello everyone. Oh, it's not Cool. Hello everyone. Oh, it's not significantly fewer. I guess it's significantly fewer. I guess it's significantly fewer. I guess it's slightly significant. Okay, … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 00:
49 | actually takes time. So I I I I mean it it takes at least a couple of hours uh it takes at least a couple of hours uh it takes at least a couple of hours uh on GPU. on GPU. on GPU. Do not even try to … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 01:
21 | sponsor is going to give us a guest lecture tomorrow. So, if you are going lecture tomorrow. So, if you are going lecture tomorrow. So, if you are going to be using the sponsor credit, I think to be u… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 01:
58 | with cloud and it's it's going to work I'm sure. Um and I already tested so so I'm sure. Um and I already tested so so I'm sure. Um and I already tested so so it should work. And uh the Oh yes, and it… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 02:
41 | for the mini class than the semester for the mini class than the semester class. So I think you guys are going to class. So I think you guys are going to class. So I think you guys are going to be mor… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 03:
23 | anything. Okay, you don't you do not need to submit any forms. Like it's just need to submit any forms. Like it's just need to submit any forms. Like it's just like basically the the the the thing is … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 04:
04 | obviously I cannot grade you. Uh but if you want to say like sit in a class you want to say like sit in a class you want to say like sit in a class which are you already did um and or uh which are you… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 04:
38 | first quiz next next class is just going to be the things that we have learned to be the things that we have learned to be the things that we have learned throughout this week. There will be no throug… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 05:
17 | distribution of the data so that we can actually draw new samples from it. Yeah. actually draw new samples from it. Yeah. actually draw new samples from it. Yeah. And we have looked at a couple of And… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 06:
06 | you try you try to learn an encoder so that it can encode it into some latent that it can encode it into some latent that it can encode it into some latent space. Uh and then like it is an space. Uh a… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 06:
47 | trying to make sure that this encoding trying to make sure that this encoding this notion of of encoding is the same this notion of of encoding is the same this notion of of encoding is the same for b… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 07:
33 | data. Okay. So this is what we can data. Okay. So this is what we can design but what we do not have is uh design but what we do not have is uh design but what we do not have is uh basically the the a… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 08:
21 | try to make sure that um the encoder and try to make sure that um the encoder and decoder are induced by the same um like decoder are induced by the same um like decoder are induced by the same um lik… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 09:
09 | You multiply by something and divide it by something and then you get a equal by something and then you get a equal by something and then you get a equal sign, right? Uh and basically this thing sign,… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 10:
00 | if your function is con concave then the the sign flips. So here the log function the sign flips. So here the log function the sign flips. So here the log function is a concave function. So that's why… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 10:
56 | remember like this thing like we do not have a sampler of x, right? So we just have a sampler of x, right? So we just have a sampler of x, right? So we just we need a sampler over z that is like we ne… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 11:
36 | anyone take a guess of like what what kind of parameterization that we're kind of parameterization that we're kind of parameterization that we're going to use here? going to use here? going to use her… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 12:
28 | parameterize Q Z uh Z over uh Z given parameterize Q Z uh Z over uh Z given accent also a gausian also gausian right uh so also a gausian also gausian right uh so basically it just just you just choos… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 13:
15 | of it to be a diagonal gausian then we can literally just write out the par the can literally just write out the par the can literally just write out the par the parameters of the distribution like th… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 14:
05 | okay. Okay. So because we have this and both of them are Gaussian. Uh so that both of them are Gaussian. Uh so that both of them are Gaussian. Uh so that this KL actually has a closed form. So this KL… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 14:
48 | decoder? Anyone want to take a guess? Maybe a gian mixture more because like Maybe a gian mixture more because like gosh is more. >> Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. >> Yeah. Yeah. Yeah. Yeah. Yeah… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 15:
58 | and then uh it's uh magically it's uh magically it's uh magically something that that is basically L2. So something that that is basically L2. So something that that is basically L2. So that's why it'… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 17:
04 | genes is like expressive. So basically they have some like lazy genes. This is they have some like lazy genes. This is they have some like lazy genes. This is kind of like that. So you you are kind of… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 17:
53 | him? Yeah. him? Yeah. >> Oh, okay. Okay. That's okay. Go ahead. >> Oh, okay. Okay. That's okay. Go ahead. >> Oh, okay. Okay. That's okay. Go ahead. >> Is there variance of the uh the Q >> Is there var… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 18:
44 | >> Two he explained to me. One is that u if it is not a dis a distribution progress it is not a dis a distribution progress it is not a dis a distribution progress take modeling it is a one to one map… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 19:
38 | Also, by the way, they're like this is like VAE is not the only way to do like VAE is not the only way to do like VAE is not the only way to do latent variable like and and actually latent variable li… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 20:
28 | calculate uh the elbow loss and then you calculate uh the elbow loss and then you uh optimize it. This is how you train a uh optimize it. This is how you train a uh optimize it. This is how you train … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 21:
15 | talk about but like they they're also talk about but like they they're also likelihood based and we can also do a likelihood based and we can also do a likelihood based and we can also do a likelihood… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 22:
07 | into a more complicated one. So this is into a more complicated one. So this is like sort of like the general underlying like sort of like the general underlying like sort of like the general underlyi… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 23:
23 | from the from from the audience. H this column. Hello, guys. H Which one H this column. Hello, guys. H Which one H this column. Hello, guys. H Which one do you prefer? I'll let you guys pick. do you p… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 24:
14 | >> Okay. So, like, uh, yeah. So, for Gans, it's very difficult to train, right? it's very difficult to train, right? it's very difficult to train, right? Basically, that's what you said. Basically, th… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 25:
10 | saying right like they cannot >> you like basically is kind of a >> you like basically is kind of a >> you like basically is kind of a continuous continuous continuous >> um so for g for for GANs espe… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 25:
57 | >> Uh VAES are going to kind of take the >> Uh VAES are going to kind of take the average of a distribution because average of a distribution because average of a distribution because they're enforcin… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 27:
01 | be able to calculate stuff one by one be able to calculate stuff one by one and for text that's okay because text is and for text that's okay because text is and for text that's okay because text is s… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 27:
35 | break things up that everything is going break things up that everything is going to be really really slow, right? So say to be really really slow, right? So say to be really really slow, right? So sa… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 28:
17 | from Reddit seven years ago are like why are VA so blurry? Yeah, that's because are VA so blurry? Yeah, that's because are VA so blurry? Yeah, that's because uh you know uh vaes are exactly right uh y… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 29:
05 | average uh data all the time and GANs are extremely extremely difficult to are extremely extremely difficult to are extremely extremely difficult to train. Uh, so even though it works train. Uh, so ev… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 29:
48 | something called mode collapse which something called mode collapse which just means that like basically it's sort just means that like basically it's sort just means that like basically it's sort of … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 30:
33 | you sample a noise from a noise distribution like Gausian and then you distribution like Gausian and then you distribution like Gausian and then you apply some sort of like magical training apply some… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 31:
14 | Do you think that's easier? Yes, it is Do you think that's easier? Yes, it is easier. This is a healthier type of easier. This is a healthier type of easier. This is a healthier type of magic. It's no… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 32:
01 | that we take from a complete noise and that we take from a complete noise and then we gradually try to dn noiseise it then we gradually try to dn noiseise it then we gradually try to dn noiseise it so… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 32:
39 | is this is all makes sense. All gonna make sense. We we just just just channel make sense. We we just just just channel make sense. We we just just just channel your inner gausian and everything's you… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 33:
23 | We're going to talk about it or or like does it actually need to be scaled? does it actually need to be scaled? does it actually need to be scaled? We're going to talk about it next class. We're going… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 34:
10 | distribution by their mean and their variance right this is basic or or yeah variance right this is basic or or yeah variance right this is basic or or yeah so this is what's happening so this is what… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 35:
04 | >> great question in general? How big is t? >> great question in general? How big is t? The this later on the the model that The this later on the the model that The this later on the the model that w… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 35:
50 | I shall hope so because this is literally the first step that how we literally the first step that how we literally the first step that how we develop the elbow from VAE as well, develop the elbow fro… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 36:
44 | joint from time zero to time t. So we're just going to break things up into just going to break things up into just going to break things up into things that we know. So first of all, we things that w… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 37:
30 | you can turn the join distribution into a chain of the the into a chain of like a chain of the the into a chain of like a chain of the the into a chain of like the your current given the previous the … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 38:
24 | basically we can get yeah anyway we can turn we can turn this yeah anyway we can turn we can turn this yeah anyway we can turn we can turn this Q here into a combination of the three Q here into a com… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 39:
22 | something is equal to the sum of log of this thing minus this thing plus this this thing minus this thing plus this this thing minus this thing plus this thing minus the next thing and then it's thing… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 40:
19 | Yeah. A plus a plus a reconstruction Yeah. A plus a plus a reconstruction laws. laws. laws. Do we do we find this familiar? No way we don't find this familiar. Do No way we don't find this familiar. D… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 41:
22 | doesn't work that well. Uh so is is there any way that we can make this work there any way that we can make this work there any way that we can make this work because that whole thing is just too beca… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 42:
14 | thing is Markoff, right? Which means that at each step, it just adds a very that at each step, it just adds a very that at each step, it just adds a very small amount of noise to the previous small am… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 43:
05 | you absolutely could now you have ddpm you absolutely could now you have ddpm so if you if if there's another slide so if you if if there's another slide so if you if if there's another slide that you… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 43:
50 | it so we just fix it right we just have it so we just fix it right we just have a fixed noising schedule that each time a fixed noising schedule that each time a fixed noising schedule that each time … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 44:
38 | right? So one gausian plus another gausian plus t gausian is still a gausian plus t gausian is still a gausian plus t gausian is still a gausian. So instead of like at in the gausian. So instead of li… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 45:
24 | both Q and P to be the same gausian. So both Q and P to be the same gausian. So it's it's going to be perfect every it's it's going to be perfect every it's it's going to be perfect every time. Doesn'… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 46:
16 | know in closed form the mean and the variance of this particular uh uh variance of this particular uh uh variance of this particular uh uh distribution that we're going to use. distribution that we're… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 47:
00 | this this variance based on the variance that you choose for the forward process. that you choose for the forward process. that you choose for the forward process. But basically what we can do is we c… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 47:
53 | this mean versus the mean that you this mean versus the mean that you predict Any predict Any predict Any question? Nope. Alrighty. Nope. Alrighty. All right. Now we have everything. All right. Now we… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 48:
48 | weight you should give to each based on your your your time step. So this is the your your your time step. So this is the your your your time step. So this is the reparameter same reparameterization r… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 49:
48 | Why is this a good thing you may ask? Why is this a good thing you may ask? This is so complicated. Why do we want This is so complicated. Why do we want This is so complicated. Why do we want this? I… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 50:
54 | uh of your that that can represent your prediction. And then here is why you can prediction. And then here is why you can prediction. And then here is why you can directly predict the noise. All right… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 51:
50 | literally they simplify into a L2 loss between the the noise that you're adding between the the noise that you're adding between the the noise that you're adding to your sample and the predicted noise… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 52:
44 | because the the the noise you predicted because the the the noise you predicted is literally how much noise should I d is literally how much noise should I d is literally how much noise should I d noi… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 53:
27 | the the the the expression for the predicted mean of that is given by XT. predicted mean of that is given by XT. predicted mean of that is given by XT. So your previous time your your current So your … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 54:
20 | step. You're not trying to jump from to x0 yet. Uh in order to jump to x0 x0 yet. Uh in order to jump to x0 x0 yet. Uh in order to jump to x0 there's actually a paper about it. And there's actually a … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 55:
00 | plus a gausian noise. So this is still this this distribution. Okay, this this distribution. Okay, this this distribution. Okay, any other questions? Cool. Erh anyhow anyhow so that's it for Cool. Erh… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 55:
53 | then you try to like basically see if the noise that you predict is accurate the noise that you predict is accurate the noise that you predict is accurate uh in comparison to the the ground truth uh i… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 56:
31 | And then like 10,000 steps later, you And then like 10,000 steps later, you get a sample. Make sense? Make sense? Cool. Amazing. And and the model is Cool. Amazing. And and the model is Cool. Amazing.… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 57:
25 | latent let's just say and then later we're trying to map this latent this we're trying to map this latent this we're trying to map this latent this gausian uh into the a reconstruction of gausian uh i… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 58:
18 | X >> no >> no >> no >> well well basically it doesn't depend on X in well basically it doesn't depend on X in the sense that it's not like the sense that it's not like the sense that it's not like it … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 59:
15 | >> This will be closer to this one like you >> This will be closer to this one like you can do interpolation which has an can do interpolation which has an can do interpolation which has an effect. ef… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 60:
02 | should be an autoenccoder which means should be an autoenccoder which means that uh it it should have some that uh it it should have some that uh it it should have some compression actually. So basica… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 60:
44 | this is not very clear anymore and it's because um it's like the the noise and because um it's like the the noise and because um it's like the the noise and the data are of the same dimension. So the … |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 61:
29 | called a unit. Okay. Uh so exactly what it is is basically you have some input it is is basically you have some input it is is basically you have some input like input image right and then you are lik… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 62:
16 | basically you are downsampling and upsampling your uh your image so that upsampling your uh your image so that upsampling your uh your image so that like basically you sort of covers both like basical… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 63:
10 | where input and outputs are about the where input and outputs are about the same dimensionality same dimensionality same dimensionality um spatially. Uh but basically later on um spatially. Uh but bas… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 63:
57 | because uh we because we have this like rep like uh we because we have this like rep like uh we because we have this like rep like this way to represent uh the the noisy this way to represent uh the t… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 64:
43 | literally just plug in this thing into your mean and then you you should be your mean and then you you should be your mean and then you you should be able to get a nicely formulated closed able to get… |
cmu-10799-diffusion-flow | L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 65:
25 | we should now know what diffusion model is. And in this class, we basically just is. And in this class, we basically just is. And in this class, we basically just derive the diffusion model from from … |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 00:
02 | Now that we have seen the probabilistic Now that we have seen the probabilistic length of generative AI, let us actually length of generative AI, let us actually length of generative AI, let us actual… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 00:
50 | distribution space. Let's say this is distribution space. Let's say this is noise and noise can live anywhere, noise and noise can live anywhere, noise and noise can live anywhere, right? So we start … |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 01:
39 | and what good will this do to me if I and what good will this do to me if I start from noise. Well, look at this start from noise. Well, look at this start from noise. Well, look at this video, right?… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 02:
29 | this and we sequentially go on adding this and we sequentially go on adding noise. We sequentially go on adding noise. We sequentially go on adding noise. We sequentially go on adding noise until it b… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 03:
36 | I will train a machine learning model to predict how much noise was added to predict how much noise was added to predict how much noise was added to predict how much noise was added. Why to predict ho… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 04:
23 | step we have a model which predicts how much noise has been added to each step. much noise has been added to each step. much noise has been added to each step. So we have to train an ML model which So… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 05:
13 | So let's say if you have if you want to So let's say if you have if you want to predict images which look like this predict images which look like this predict images which look like this panda bear p… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 06:
01 | slowly. If you do this experiment at slowly. If you do this experiment at home and if you stir this fluid very home and if you stir this fluid very home and if you stir this fluid very fast and if you… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 07:
17 | diffusion dnoising and noising is also diffusion dnoising and noising is also called as forward diffusion. So this called as forward diffusion. So this called as forward diffusion. So this same intuit… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 07:
59 | So again to summarize the way diffusion So again to summarize the way diffusion for images works is that we take a for images works is that we take a for images works is that we take a random image fr… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 08:
53 | how to reduce the noise. how to reduce the noise. By doing this repeatedly the model can By doing this repeatedly the model can By doing this repeatedly the model can transform any noise to a sample t… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 09:
45 | That's the beauty of diffusion process. So there are certain key characteristics So there are certain key characteristics So there are certain key characteristics of diffusion. So there are key charac… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 10:
41 | here we use a machine learning model to here we use a machine learning model to predict this noise. Um and to make it predict this noise. Um and to make it predict this noise. Um and to make it more s… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 11:
28 | generate. So these three are the key So these three are the key So these three are the key characteristics of a diffusion model. characteristics of a diffusion model. characteristics of a diffusion mo… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 12:
37 | moving in a very uh straightforward manner for image generation and people manner for image generation and people manner for image generation and people were happy. There were companies which were hap… |
diffusion-lm-vizuara | L5: Lecture 4: How Diffusion Models Work for Image Generation | 14:
14 | And we are just going to start looking And we are just going to start looking at this field right now. I hope the at this field right now. I hope the at this field right now. I hope the motivation for… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 00:
05 | Hello everyone and welcome to the third Hello everyone and welcome to the third lecture of this course principles of lecture of this course principles of lecture of this course principles of diffusion… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 01:
10 | hidden factors or the factors which hidden factors or the factors which affect the variation in the data. For affect the variation in the data. For affect the variation in the data. For example, if we… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 02:
12 | transform them into a Latin space with only two dimensions X and Y data. only two dimensions X and Y data. only two dimensions X and Y data. The Latin space the intuition behind The Latin space the in… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 03:
15 | it's also called as a style variable it's also called as a style variable because it captures the style which is because it captures the style which is because it captures the style which is inherent … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 04:
14 | you get the image out of it. That is this machine is called as the decoder. this machine is called as the decoder. this machine is called as the decoder. So the decoder takes in the latent So the deco… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 05:
25 | when we map the word hello into the areas of the Latin space, we don't just areas of the Latin space, we don't just areas of the Latin space, we don't just map it to one single value, but we map map i… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 06:
28 | we saw the Latin space after the training is completed. It looks training is completed. It looks training is completed. It looks something like this. something like this. something like this. Now Now … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 07:
29 | space. You have a decoder which maps it back. How do you train the variation back. How do you train the variation back. How do you train the variation autoenccoder? What's the process that autoenccode… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 08:
32 | distribution. So it tries to move the distribution in So it tries to move the distribution in So it tries to move the distribution in the latin space to a gshian the latin space to a gshian the latin … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 09:
31 | of the students of your class and let's of the students of your class and let's look at this factor which is neatness. look at this factor which is neatness. look at this factor which is neatness. Oka… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 10:
19 | distributions that you see in in reality and that's what people try to capture and that's what people try to capture and that's what people try to capture over here. over here. over here. So mathemati… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 11:
19 | understand uh the two types of losses understand uh the two types of losses which we are encountering here. Okay. So, uh the lower bound is called Okay. So, uh the lower bound is called as elbow which… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 12:
27 | P of X is not possible to calculate. P of X is not possible to calculate. It's intractable It's intractable It's intractable because you need to calculate an because you need to calculate an because y… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 13:
24 | say that is the recipe written in standard language. Does it follow a standard language. Does it follow a standard language. Does it follow a distribution which is shown in yellow? distribution which … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 14:
34 | And this is something we can clearly see from this image as well. Every single from this image as well. Every single from this image as well. Every single output image appears blur compared to output … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 15:
33 | linked to the decoder which means that linked to the decoder which means that we have to train two neural networks in we have to train two neural networks in we have to train two neural networks in th… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 16:
35 | which you can see over here denoising which you can see over here denoising diffusion probabilistic models diffusion probabilistic models diffusion probabilistic models and this paper has over and thi… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 17:
42 | literature. So diffusion models existed before as So diffusion models existed before as So diffusion models existed before as well but people had not proven that they well but people had not proven th… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 18:
36 | authors have written. So for that purpose I would really recommend all of purpose I would really recommend all of purpose I would really recommend all of you to take a print out and keep it you to tak… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 19:
42 | other corner of the room, right? So it's other corner of the room, right? So it's it's it's like the it's it's like the it's it's like the particles which are uh particles which are uh particles which… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 20:
45 | of artificial intelligence. It sounded It sounded It sounded like ridiculous because how how can you like ridiculous because how how can you like ridiculous because how how can you apply this to AI, r… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 21:
44 | AI and uh we are going to see how the AI and uh we are going to see how the diffusion technique is used for diffusion technique is used for diffusion technique is used for reproducing the original dat… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 22:
43 | That's what we are going to think about because our primary intention is to because our primary intention is to because our primary intention is to reproduce the original or the true data reproduce th… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 23:
56 | uniform and noisy removes all the uniform and noisy removes all the structure in the data. structure in the data. structure in the data. What if we think of our encoder as being What if we think of ou… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 24:
58 | disappear. When people look at the final image, they have no idea that this came image, they have no idea that this came image, they have no idea that this came from Batman. This could have this this … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 25:
58 | step. First we will add some noise. Then step. First we will add some noise. Then we'll add another noise and then slowly we'll add another noise and then slowly we'll add another noise and then slowl… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 26:
59 | am holding this bottle in front of me. I take an image of this and I want take an image of this and I want take an image of this and I want something which can transform this image something which can… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 27:
52 | see all of these individual grids. They are one one pixel each and every single are one one pixel each and every single are one one pixel each and every single pixel holds some value. For example, pix… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 29:
18 | obviously the value is not going to be 0.5. The value is going to be different 0.5. The value is going to be different 0.5. The value is going to be different than 0.5. It's either going to be than 0.… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 30:
03 | the same deviation of of beta 1. and the same deviation of of beta 1. and then we'll again sample from it. then we'll again sample from it. then we'll again sample from it. So we will continue to do t… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 31:
06 | this. You can see that it's not this. You can see that it's not completely noise but it's becoming completely noise but it's becoming completely noise but it's becoming noisier, right? It's it's slowl… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 32:
07 | gshian transition kernel and you do it several times. several times. several times. So this is the process which is called So this is the process which is called So this is the process which is called… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 33:
16 | Let's try to understand. Let's try to understand. Remember that we are preserving the mean Remember that we are preserving the mean Remember that we are preserving the mean value of the pixels. The me… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 34:
31 | from a gshian which with a mean which is from a gshian which with a mean which is slightly scaled down from this mean. slightly scaled down from this mean. slightly scaled down from this mean. So this… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 36:
03 | down the mean, we have made sure that down the mean, we have made sure that the mean of every single pixel goes down the mean of every single pixel goes down the mean of every single pixel goes down f… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 36:
59 | bottles into pure noise. bottles into pure noise. And that happens for any data that you And that happens for any data that you And that happens for any data that you feed into the encoder. feed into … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 38:
02 | from where any image in the world can be generated. generated. generated. So essentially we are looking at So essentially we are looking at So essentially we are looking at something which is at the h… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 38:
51 | >> This is given by this. The first image I'm going to denote as X0. The second I'm going to denote as X0. The second I'm going to denote as X0. The second image I'll denote as X1 and so on. Now image… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 39:
48 | also called as noise schedule. So beta is what adds noise to the image. So beta is what adds noise to the image. So beta is what adds noise to the image. Uh and we have different betas because Uh and … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 40:
38 | uh find out values for your alphas? And this is something which uh even I And this is something which uh even I And this is something which uh even I was thinking for some time how are the was thinkin… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 41:
45 | such that alpha square + beta square always becomes one. That is how you choose these alphas and That is how you choose these alphas and bet alphas. After you first choose the bet alphas. After you fi… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 43:
21 | Okay. So in the first step we import our libraries. We are going to use numpy and libraries. We are going to use numpy and libraries. We are going to use numpy and uh mattplot lab for this example. So… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 44:
21 | after 100 different steps. Maybe this number you might not even Maybe this number you might not even Maybe this number you might not even require to be this high, but people require to be this high, b… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 45:
21 | our discussion as well. So let me run this. this. this. Okay. Now we do the u Okay. Now we do the u Okay. Now we do the u step-by-step forward diffusion process. step-by-step forward diffusion process… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 46:
15 | So it's it's it's it's a simple trick So it's it's it's it's a simple trick which is to say that you which is to say that you which is to say that you uh keep the original value or rather you can say … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 47:
26 | before here the mu gets changed by before here the mu gets changed by alpha* alpha* alpha* alpha* the original pixel value and the alpha* the original pixel value and the alpha* the original pixel val… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 48:
24 | total of 64x 64 pixels and for every total of 64x 64 pixels and for every single pixel we choose a random variable single pixel we choose a random variable single pixel we choose a random variable uh … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 49:
11 | there it really helps if we go step by step right you can't generate an image step right you can't generate an image step right you can't generate an image directly from noise that is too much directl… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 49:
56 | structure disappears and it becomes uniform. So it satisfies our original uniform. So it satisfies our original uniform. So it satisfies our original criteria that if we want this to be criteria that … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 50:
55 | want to go directly from here to here. want to go directly from here to here. So there is a way to uh write the mean So there is a way to uh write the mean So there is a way to uh write the mean and t… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 51:
52 | transition. If I give this as the output, then write down the effective output, then write down the effective output, then write down the effective mean and the effective variance for this mean and th… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 52:
37 | disappear and the second tenant is the structure should become uniform with structure should become uniform with structure should become uniform with time. This is exactly what we did with time. This … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 53:
31 | that we are going to handle right now. So the the first intuition probably So the the first intuition probably So the the first intuition probably tells us that to go from noise to the tells us that t… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 54:
26 | this image to noise is is is maybe scaling the mean by alpha and maybe scaling the mean by alpha and maybe scaling the mean by alpha and adding beta. Why don't we just upscale adding beta. Why don't w… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 55:
29 | the original data. So this will become clear as as we go So this will become clear as as we go So this will become clear as as we go along. Let's let's move along. along. Let's let's move along. along… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 56:
26 | right. So you can see that this is not like a you can see that this is not like a you can see that this is not like a one-time denoising step like we did for one-time denoising step like we did for on… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 57:
31 | And that probability should be very And that probability should be very high, right? because we want our decoder high, right? because we want our decoder high, right? because we want our decoder to as… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 58:
30 | So if we maximize that lower bound it So if we maximize that lower bound it also means that we are maximized the also means that we are maximized the also means that we are maximized the true objectiv… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 59:
34 | before, we are going to get pure noise only. So there is no doubt that this is only. So there is no doubt that this is only. So there is no doubt that this is going to be pure noise. So this is going … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 60:
26 | So this is saying that I want to So this is saying that I want to minimize the kale divergence between minimize the kale divergence between these two probabilities. these two probabilities. these two … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 61:
32 | what we are trying to predict how do we actually go from x2 to x1 this is what actually go from x2 to x1 this is what actually go from x2 to x1 this is what our neural network will predict and our our… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 62:
24 | true data. So if we achieve this, then we are anyways going to achieve a we are anyways going to achieve a we are anyways going to achieve a reconstruction which is that is good. reconstruction which … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 63:
11 | So let's let's take an example and if So let's let's take an example and if you are bothered by the word posterior you are bothered by the word posterior you are bothered by the word posterior don't w… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 64:
08 | rain fell on this paper rain fell on this paper how did it look like? how did it look like? how did it look like? And that's very hard to tell because And that's very hard to tell because And that's v… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 65:
00 | maybe if I just remove some noise it will look something similar to meat. will look something similar to meat. will look something similar to meat. This will look something similar to me This will loo… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 66:
00 | you go from X3 to X2? That is the question. Okay, That is the question. Okay, That is the question. Okay, now now now you have access to x3 and you also have you have access to x3 and you also have yo… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 67:
05 | fine but if we are able to predict this why can't we just why can't we just why can't we just match this directly to this? If if we match this directly to this? If if we match this directly to this? I… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 68:
18 | by the way this is one path that you by the way this is one path that you take from the noise to the data data. take from the noise to the data data. take from the noise to the data data. This is anot… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 69:
14 | understand first how how this posterior is is predicted. is is predicted. is is predicted. It turns out that this this reverse It turns out that this this reverse It turns out that this this reverse p… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 70:
29 | How much importance should I place on X3 How much importance should I place on X3 and how much importance should I place and how much importance should I place and how much importance should I place o… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 71:
26 | process. And I think we do have a graph which talks about it. Yeah. Yeah. Here. which talks about it. Yeah. Yeah. Here. which talks about it. Yeah. Yeah. Here. So here you see that uh the weightage on… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 72:
28 | So, so you should read the time as this this to this. So, here we have the this to this. So, here we have the this to this. So, here we have the complete noisy image complete noisy image complete nois… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 73:
23 | as a3 which is a function of all these as a3 which is a function of all these variances. variances. variances. So basically you go from uh the original So basically you go from uh the original So basi… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 74:
47 | the original image to the final image in 100 steps. We have already seen how to 100 steps. We have already seen how to 100 steps. We have already seen how to uh write down this forward diffusion uh wr… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 75:
48 | alphas and betas which is mentioned in the paper. So I have deliberately stayed the paper. So I have deliberately stayed the paper. So I have deliberately stayed away from it. I don't want to get too … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 76:
53 | to 49 48 47 and finally you move to to 49 48 47 and finally you move to the original image. So, so it actually the original image. So, so it actually the original image. So, so it actually works right… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 77:
45 | their codes on GitHub. So this alpha actually represents alpha So this alpha actually represents alpha So this alpha actually represents alpha square in our notation and this beta square in our notati… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 78:
37 | this beta is it it represents beta this beta is it it represents beta square which is the variance in our square which is the variance in our square which is the variance in our notation and alpha rep… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 79:
28 | the image from noise. So the original the image from noise. So the original image will not be given to us. image will not be given to us. image will not be given to us. So basically we want to uh pred… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 80:
32 | Okay. So now we have two gshian distributions. First goshian distributions. First goshian distributions. First goshian distribution we know the mean and distribution we know the mean and distribution … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 81:
28 | which we have predicted over here. >> [snorts] >> [snorts] >> So now it boils down to a simple uh >> So now it boils down to a simple uh >> So now it boils down to a simple uh algebraic manipulation. … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 82:
23 | really know. And now if you apply this formula mu of 5 - mu² formula mu of 5 - mu² formula mu of 5 - mu² what you see is you want to basically what you see is you want to basically what you see is you… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 83:
38 | uh the reverse transition process will be same as that of the true posterior be same as that of the true posterior be same as that of the true posterior because then you have the same because then you… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 84:
32 | it can be written as a function of the it can be written as a function of the original image original image original image and noise. and noise. and noise. Remember this was the homework which I Remem… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 85:
34 | uh how much noise we are basically adding at each step. adding at each step. adding at each step. Okay. So uh but for now let's let's keep Okay. So uh but for now let's let's keep Okay. So uh but for … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 86:
55 | So So this is what this means. So x0 can in fact be written in as in in this way. in fact be written in as in in this way. in fact be written in as in in this way. And in the paper what they do is sin… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 87:
53 | >> Now you see the uh simplicity of this. >> Now you see the uh simplicity of this. What this essentially means is that What this essentially means is that What this essentially means is that what I n… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 88:
51 | we have imparted at each transition. Once we have this noise level, we can Once we have this noise level, we can Once we have this noise level, we can actually calculate the the the images at every si… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 90:
09 | equation which which they get uh you know and here you see we finally uh you know and here you see we finally uh you know and here you see we finally have epsilon minus epsilon theta whole have epsilo… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 91:
17 | diffusion models and and why they work. diffusion models and and why they work. So uh please take a print out and go So uh please take a print out and go So uh please take a print out and go through a… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 92:
13 | encoder as a goshian transition kernel with a specific mean and variance. And with a specific mean and variance. And with a specific mean and variance. And we have different iterations which go we hav… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 93:
05 | first two terms are exactly same like we saw in VAS. The regularization term is saw in VAS. The regularization term is saw in VAS. The regularization term is not important here because we already not … |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 93:
54 | posterior in a very tractable way. And we saw that it can be written as a we saw that it can be written as a we saw that it can be written as a gshian with a mean and a variance. The gshian with a mea… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 94:
42 | more, we we found out that at the heart of it, we want to make sure that the of it, we want to make sure that the of it, we want to make sure that the original image true data which original image tru… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 95:
40 | in the forward diffusion process. in the forward diffusion process. And this this makes a lot of sense. And this this makes a lot of sense. And this this makes a lot of sense. That's why the name dino… |
diffusion-principles-vizuara | L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 96:
26 | please let me know and just just sit down and go through and just just sit down and go through and just just sit down and go through this homework problem where you find the this homework problem wher… |