概念

VAE Encoder

— 把 x 映成"区域 q(z|x)"而非单点;diffusion 的 latent space 祖宗。

很多人第一次见到 VAE 都以为它就是「autoencoder + KL 正则项」——把瓶颈层多套一层惩罚,让 latent code 长得"更规整"一点。这是教科书最常见的误解。VAE encoder 真正的不同是:它输出的不是一个点,而是一个分布 $q_\phi(z|x)$,这样你才能从先验 $\mathcal{N}(0, I)$ 里采 z 去解码出"从没见过、但风格合理"的新样本。换言之,AE 学的是压缩,VAE 学的是采样。

Vizuara 的 Dr. Raj 在《Principles of Diffusion Models》Lecture 2 用一个特别朴实的比喻把这件事讲清楚——古董魔术机。问题是这样的:班里 100 个学生每人写一次 "hello",你要造一台机器,按一下按钮就吐出一张"像某个学生写的 hello"。注意"像某个学生"——不是复制某张已有图片,而是从手写风格分布里抽一张新样本。这才是 generative modeling 的核心问题,跟 AE 的"重建训练集"完全不同。

「想象班里 100 个学生,每人写 hello。给我造一台机器——按一下吐一个『某学生写的 hello』。」—— Vizuara L9 @02:19

Dr. Raj 在 chunk #14 进一步把这个机器拆开:每个学生的字之所以长那样,是因为有一组隐藏因素——笔压、倾斜角、字宽、书写速度、整洁度。他把这五个因素叫做"secret recipe"。如果你能猜到 secret recipe,就能反推手写。这五个数字就是latent variable $z$。VAE encoder 的工作,是从 $x$(一张 hello 图)里反推 $z$ 的分布——为什么是分布?因为光看一张图你不能笃定"该学生笔压一定是 0.5",只能说"在 0.3 到 0.7 之间,均值 0.5、标准差 0.2 比较像"。这个"均值 + 标准差"就是 $q_\phi(z|x)$。

CMU 10-799 的 Kelly 在 Lecture 1 给了第二个比喻:把生成模型想成"对一群学生身高的概率分布"——你有几个朋友的身高样本,但不知道整个学校的真实分布。生成模型的目的就是从样本推回分布,然后从分布采新样本。两个比喻指向同一件事:generative modeling 是"分布近似 + 分布采样"的组合,不是单纯的记忆。VAE 是其中一种实现,给"采样"配上了一个可微的、用神经网络参数化的 latent 通道。

在训练里,encoder 对应一段神经网络:输入 $x$、输出 $\mu_\phi(x), \sigma_\phi(x)$,从 Gaussian 采 $z = \mu + \sigma \cdot \epsilon$(reparameterization trick,见 §Deep dive),送进 decoder 算重建 + KL。整套架构最早由 Kingma & Welling (2014) 在 Auto-Encoding Variational Bayes 一文里写下来。

🟡 教材外补充:

Vizuara 把 VAE 当成"独立目标"来教;但在 2026 年的实际工程里,VAE encoder 几乎只剩一种用法——给 Latent Diffusion (Stable Diffusion) 当"感知压缩器"。它先在大规模图像上预训练,训完冻住,然后扩散模型在它的 latent 上跑去噪。所以现在工程师看到"VAE encoder"四个字,脑子里浮现的不是"采样新数据的生成模型",而是"把 512×512×3 像素压到 64×64×4 latent 的固定函数"。Vizuara 的 hello 机器讲的是 VAE 的初心;现代用法是它的转世。两个视角都要熟。

关键改动:输出分布,不是点

普通 AE 的 encoder 是一个确定性映射 $f_\theta: x \mapsto z$,一张图进、一个向量出。VAE encoder 是一个随机映射 $g_\phi: x \mapsto (\mu, \sigma) \mapsto q_\phi(z|x) = \mathcal{N}(\mu, \sigma^2)$,输出的是一个分布的参数,真正用到 $z$ 时要从这个分布里采一下。这一步"采样"不是工程上的可选项,它让 KL 项有意义,也让训练目标统一在 ELBO 框架下: $$\log p(x) \geq \mathbb{E}_{q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}\!\left(q_\phi(z|x) \,\|\, p(z)\right)$$ 右边就是 VAE 的目标函数。第一项是重建("采一个 z 解码,看离 x 多远"),第二项是 KL,把后验 $q_\phi(z|x)$ 拉向先验 $p(z) = \mathcal{N}(0, I)$。最大化 ELBO 等价于:在不知道真实 $p(x)$ 的前提下,从下方把 log-likelihood 撑上去。

Why 重建项?没有它,encoder 可以摆烂——直接输出 $\mu = 0, \sigma = 1$,KL = 0,但 $z$ 跟 $x$ 无关,decoder 解出来全是均值脸。重建项强制 $z$ 携带 $x$ 的信息。

Why KL 项?没有它,encoder 可以把每张图映射到 latent 空间里一个孤立的小点($\sigma \to 0$),整个 latent space 全是洞,从先验采一个 $z$ 几乎肯定落到洞里、decoder 解出垃圾。KL 把所有 $q_\phi(z|x)$ 整体推向 标准正态,让 latent 空间被"填满"、连续无洞。这是 VAE 能"从先验采样新数据"的关键。

Reparameterization trick(前向指针 → 见单独条目)

采样 $z \sim \mathcal{N}(\mu, \sigma^2)$ 是不可导的——你不能对"随机数"求梯度。Kingma 的 trick:把随机性外置——$\epsilon \sim \mathcal{N}(0, I)$,然后 $z = \mu + \sigma \cdot \epsilon$。这样 $\mu, \sigma$ 是可微的,$\epsilon$ 是外部噪声、不需要梯度,整条计算图就 backprop 通了。工程实现上还有个稳定性细节:

工程细节:网络通常输出 $\log \sigma^2$ 而不是 $\sigma$,原因有二——(1) $\sigma > 0$ 的约束自动满足(指数化);(2) 数值上 $\log \sigma^2$ 在 $\sigma$ 趋于 0 时表现稳定,不会爆梯度。
x(hello) μ_φ(x) log σ²_φ(x) z = μ+σε ε ~ N(0, I) x̂ encoder g_φ decoder p_θ(x|z)

KL 项展开(解析解)

当 $q = \mathcal{N}(\mu, \sigma^2)$、$p = \mathcal{N}(0, 1)$ 时,KL 有闭式: $$D_{\mathrm{KL}}\!\left(\mathcal{N}(\mu, \sigma^2) \,\|\, \mathcal{N}(0, 1)\right) = \tfrac{1}{2}\left(\mu^2 + \sigma^2 - 1 - \log \sigma^2\right)$$ Why 这个形式?KL 衡量两个分布的差距:

  • $\mu^2$ 惩罚均值偏离 0;
  • $\sigma^2 - \log \sigma^2 - 1$ 是 $\sigma^2$ 在 1 处的极小值(求导 $1 - 1/\sigma^2 = 0 \Rightarrow \sigma^2 = 1$),惩罚方差偏离 1(无论太大太小)。

多维情形对每个维度独立求和。代码里通常是一行:kl = 0.5 * torch.sum(mu**2 + logvar.exp() - 1 - logvar)。注意 reduction='sum' 还是 'mean'——前者会让 KL 权重隐含地随 latent 维数线性增长,是常见的训练失误源。

encoder 架构层面:实际网络长什么样

在 MNIST/CelebA 这种小图上,encoder 通常是 3-5 层 CNN(每层下采样 2 倍)+ flatten + 两个并行的全连接层分别输出 $\mu$ 和 $\log \sigma^2$。latent 维度典型选 16-128。在 Stable Diffusion 1.x 里,encoder 是 ResNet-style 的下采样网络,输入 512×512×3,输出 64×64×4(下采样 8 倍,通道数从 3 升到 4,再单独的卷积层产生 $\mu, \log \sigma^2$)。SD 2 / SDXL 用同样架构但训练数据和损失加权不同;Flux 用了一个更深的 VAE,latent 是 64×64×16,给扩散模型更宽的信息带宽。

Why 卷积而不是全连接?图像有空间结构——平移、局部相关。卷积的 inductive bias 让 latent 也保持空间结构(一个 64×64 的 latent 图,每个像素仍然对应原图的一个局部区域)。这对扩散模型很关键——U-Net 也是空间结构网络,二者匹配。如果用全连接 VAE encoder 把图压成 1D 向量,扩散模型就没法用 U-Net 了。

β-VAE 与 disentangling

Higgins 等 (2017) 的 β-VAE 把 KL 系数从 1 改成可调的 $\beta$: $$\mathcal{L}_\beta = \mathbb{E}_q[-\log p(x|z)] + \beta \cdot D_{\mathrm{KL}}(q(z|x) \| p(z))$$ Why? 当 $\beta > 1$,KL 压得更紧,每个 latent 维度的"信息预算"变小,网络被迫把信息分摊到不同维度上——这逼出了"disentangled latent":一个维度对应"光照",另一个对应"角度",互不耦合。代价是重建变模糊(信息总量被限制了)。$\beta < 1$ 反过来,重建更清晰但 latent 杂乱。

Posterior collapse(失败模式)

失败模式:当 decoder 太强(如 PixelCNN、LSTM、Transformer),它能仅凭 $x$ 的局部统计就把 $x$ 重建得很好,根本不需要 $z$。这时 encoder 学到 $q(z|x) \approx p(z) = \mathcal{N}(0, I)$——KL 项被精确压到 0,$z$ 不携带任何信息,VAE 退化为只有 decoder 的自回归模型。这叫 posterior collapse,文本 VAE 的经典痛点。缓解:KL annealing(前期 $\beta \to 0$,让 encoder 先学会用 $z$)、free-bits(保留 KL 下限)、限制 decoder 容量。

ELBO 完整推导(为什么是这个目标函数)

Why ELBO?我们想最大化数据似然 $\log p(x)$,但 $p(x) = \int p(x, z) dz$ 是高维积分,算不动。VAE 的做法是引入一个变分后验 $q_\phi(z|x)$(也就是 encoder),用它把 $\log p(x)$ 拆开:

$\log p(x) = \log \int p(x, z) dz = \log \int q(z|x) \cdot \frac{p(x, z)}{q(z|x)} dz$

用 Jensen 不等式($\log$ 是凹的,期望往里走):

$\log p(x) \geq \mathbb{E}_{q(z|x)}\!\left[\log \frac{p(x, z)}{q(z|x)}\right] = \mathbb{E}_q[\log p(x|z)] + \mathbb{E}_q[\log p(z) - \log q(z|x)]$

第二项 $\mathbb{E}_q[\log p(z) - \log q(z|x)] = -D_{\mathrm{KL}}(q(z|x) \| p(z))$,于是:

$\log p(x) \geq \underbrace{\mathbb{E}_q[\log p(x|z)]}_{\text{重建}} - \underbrace{D_{\mathrm{KL}}(q(z|x) \| p(z))}_{\text{KL 拉向先验}}$

右边就是 ELBO(evidence lower bound)。Why 这是下界?因为 Jensen 不等式给的就是下界——它和 $\log p(x)$ 的差恰好等于 $D_{\mathrm{KL}}(q(z|x) \| p(z|x))$(真后验和我们用的变分后验的距离)。当 $q$ 完美贴合真后验时下界紧。所以最大化 ELBO 既在抬 $\log p(x)$,也在拉近 $q$ 和真后验——两件事一石二鸟。

与 DDPM、Latent Diffusion 的关系

这是 v4 版本最关键的延伸——VAE 不是死技术,它是 diffusion 的祖宗。

  • DDPM = T 步分层 VAE。DDPM 的前向过程 $q(x_t | x_{t-1})$ 是固定的高斯(不学 encoder),反向 $p_\theta(x_{t-1} | x_t)$ 是高斯(学 decoder)。整个链条等价于 $T$ 层的 hierarchical VAE,每一层都用高斯条件分布。VAE 是 $T=1$ 的特例,DDPM 是 $T=1000$ 的极端。把 $T$ 拉开后,每步的"重建"变得极其简单(只需去一点点噪声),训练目标坍缩成 MSE,反而比 VAE 好训。
  • Latent Diffusion (Rombach 2022, Stable Diffusion)。先用 VAE encoder 把 512×512×3 像素压成 64×64×4 latent,冻住,然后扩散模型在这个 latent 上做。VAE encoder 在这里被重新定位为"感知压缩器 / perceptual compressor"——它不再负责采样,只负责把"像素冗余"挤掉,让扩散模型有低维空间可用。训练损失加 LPIPS 和 GAN 项(VQ-GAN 风格),保证 latent 在感知上是可逆的,不是统计上。
  • VQ-VAE / VQ-GAN(van den Oord 2017、Esser 2021)。把连续 latent 换成离散 codebook:encoder 输出连续向量、最近邻量化到码本里的一个 index。优势:可以接 transformer(图像 token 化)。代价:不可微,要用 straight-through estimator。Stable Diffusion 1.x 用的是连续 VAE,DALL-E 1、Parti、MagViT 用的是 VQ。
🟡 教材外补充:

2026 年的实际经验是:Stable Diffusion 的 VAE encoder 完全不是 Kingma 原版 VAE。它的 KL 权重小到几乎可以忽略($\beta \approx 10^{-6}$),$\sigma$ 接近 0,几乎是 deterministic AE;保留 VAE 框架只是为了让 latent 不至于完全坍缩到流形上。所以你在 SD 代码里看到 encoder.encode(x).latent_dist.sample()——它有一个 latent_dist,但工程上你直接用 .mode()(取 $\mu$)和 .sample() 几乎没差别。"VAE encoder 在现代 SD 里是 perceptual decompression 的固定函数,不是生成模型的一部分。"这件事 Vizuara 没明说,但对工程师很重要。

训练时机与 freeze 策略

Latent Diffusion 的标准训练 pipeline 分两阶段:

  1. 阶段一:单独训 VAE 几十万步,损失 = L1 重建 + LPIPS 感知损失 + GAN 损失(patch discriminator)+ 极小的 KL。这一阶段没有 diffusion 出场,纯粹是把"像素压缩到感知等价的 latent"这件事做扎实。
  2. 阶段二:把 VAE encoder 和 decoder 都冻住,扩散模型 U-Net 在 encoder 输出的 latent 上学去噪。冻住的原因:(a) latent 空间一旦动,整个 U-Net 等价于在变化的目标上学习,不收敛;(b) 工程上节省一倍显存。SD 3、Flux 等 2024-2026 模型仍然保留这个范式,只在 VAE 架构细节(channel 数、下采样比例)上做改进。

有意思的对比:VAE 在 DDPM 里是被替代品(DDPM 自己就是个分层 VAE,不需要单独的 encoder),VAE 在 LDM 里是基础设施(不再生成、只压缩)。同一个组件,两种用法。这种"组件功能漂移"是 diffusion 时代很常见的现象,跟早期 GAN 时代的 encoder 用途又完全不同。

下面这张图展示一个 latent 平面——每个学生对应一团 Gaussian blob($q_\phi(z|x_i)$),蓝色 z 点呼吸表示"每次 forward 从 q(z|x) 重采",浅色小点是从先验 $\mathcal{N}(0, I)$ 采的新样本,它们恰好落在某些 blob 附近——这就是生成新 hello 的瞬间。不变量:每条 edge distribution 永远是 $\mathcal{N}(\mu(x), \sigma(x)^2)$,不管 $x$ 怎么变,分布形状(高斯)不变,只有参数变。

latent 平面 z ∈ ℝ²(每个 blob = 一个学生的 q(z|x)) 学生₁ 学生₂ 学生₃ 学生₄ 学生₅ prior N(0, I) z₁ 当前采样 浅色小点 = 从先验采的新 z → decoder → 新 hello

手算:重建 + KL 一次 forward

设输入 $x = 1$(一维标量为简化);encoder 输出 $\mu = 0.5$、$\log \sigma^2 = 0.365$(即 $\sigma^2 = 1.44, \sigma = 1.2$);reparam 噪声 $\epsilon = 0.1$。则:

$z = \mu + \sigma \cdot \epsilon = 0.5 + 1.2 \cdot 0.1 = 0.62$

取 $\hat x = z$ 作 trivial decoder(实际是神经网):

重建项 $= (\hat x - x)^2 = (0.62 - 1)^2 = (-0.38)^2 = 0.1444$

KL 项 $= \tfrac{1}{2}(\mu^2 + \sigma^2 - 1 - \log \sigma^2) = \tfrac{1}{2}(0.25 + 1.44 - 1 - 0.365) = \tfrac{1}{2}(0.325) = 0.1625$

$\mathcal{L} = 0.1444 + 0.1625 = 0.3069$

反向:$\partial \mathcal{L} / \partial \mu = 2(\hat x - x) \cdot 1 + \mu = -0.76 + 0.5 = -0.26$(重建拉 $\mu$ 上去,KL 拉 $\mu$ 回 0,净效果上推 0.26);$\partial \mathcal{L} / \partial \sigma = 2(\hat x - x) \cdot \epsilon + \sigma - 1/\sigma = -0.076 + 1.2 - 0.833 = 0.291$(净效果下压 $\sigma$)。注意 $\epsilon$ 只出现在重建项的链式里,KL 项的导数与 $\epsilon$ 无关——这正是 reparam 把"随机"和"可微"分离开的妙处。

常见误区

  1. "VAE 就是 autoencoder + KL 正则项。" 反例:把 KL 系数调到 0,你拿到一个普通 AE,重建几乎完美但从先验采 z 解码出来全是噪声——因为 latent 空间全是孤立小点。VAE 的目标是采样不是压缩,KL 是核心不是装饰。VAE 重建质量通常差于 AE,这是付的"采样能力"代价。
  2. "$\sigma$ 训完都接近 1。" 反例:CelebA 上训完的 VAE,背景纯色图像 $\sigma \approx 0.1$(信息少、$z$ 取值范围可以小),人脸细节复杂图像某些维度 $\sigma \approx 1.5$(信息多、需要更大变化)。KL 只是把 $\sigma$ "平均"拉向 1,不是强制每张图都是 1。
  3. "训完 latent 空间就是 $\mathcal{N}(0, I)$。" 反例:把所有 $q_\phi(z|x_i)$ 在训练集上聚合得到 aggregate posterior $q(z) = \frac{1}{N}\sum_i q_\phi(z|x_i)$,它和先验 $\mathcal{N}(0, I)$ 通常有显著 gap——这就是 VAE 生成的样本比 GAN 模糊的根本原因。后来的 VAE 变体(VQ-VAE、NVAE、Score-based prior)都在补这个 gap。

相关概念

来源

  1. diffusion-principles-vizuara · L9 · Lecture 2 - VAE Explained From Scratch · 「hello 古董魔术机」 · (2:19)
  2. diffusion-principles-vizuara · L9 · 「secret recipe / 五个 hidden factors」 · (6:45)
  3. cmu-10799-diffusion-flow · L4 · Basics of Probabilistic & Generative Modeling · (0:02)
查看 240 条课程原文出处
CourseLectureTimeExcerpt
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 00: 02 All right, class. Class, shall we begin? All right, class. Class, shall we begin? Shall we begin? Shall we begin? Shall we begin? Welcome to 10799. Welcome to 10799. Welcome to 10799. My name is Kelly…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 01: 24 2025 will be the Italian brain rods. Uh so for those of you guys who don't know so for those of you guys who don't know so for those of you guys who don't know this is this is this is this is bombado …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 02: 23 the same underlying technology and that the same underlying technology and that is is is hello and that is we got stuck. Oh and that is and that is we got stuck. Oh and that is this uh diffuser um and…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 03: 15 implement these things um like in Python implement these things um like in Python which is the language that we use every which is the language that we use every which is the language that we use ever…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 04: 06 probably need to first answer what is a probably need to first answer what is a good generation uh image generation good generation uh image generation good generation uh image generation model right …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 04: 52 they like have a lot of like weird artifacts? Um so you probably don't artifacts? Um so you probably don't artifacts? Um so you probably don't really understand what what weird really understand what …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 05: 34 or can I use another image to you know or can I use another image to you know to specify what I want to generate? For to specify what I want to generate? For to specify what I want to generate? For ex…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 06: 09 model. Can we do that? Um so that's the second aspect. And then the third aspect second aspect. And then the third aspect second aspect. And then the third aspect is basically how fast is the generati…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 06: 45 I reduce war clock time with the same amount uh same number of network amount uh same number of network amount uh same number of network evaluation which means that like you evaluation which means tha…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 07: 31 beginnings village and where you you you beginnings village and where you you you get to like fail pretty fast and safely get to like fail pretty fast and safely get to like fail pretty fast and safel…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 08: 07 bosses throughout the way. And uh, these bosses throughout the way. And uh, these ones are not going to kill you and ones are not going to kill you and ones are not going to kill you and they're just …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 08: 52 environment try to get yourself familiar environment try to get yourself familiar with GPU training and also just build with GPU training and also just build with GPU training and also just build two …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 09: 42 Okay. And uh at the end of the class uh Okay. And uh at the end of the class uh which is the last week uh of class, it's which is the last week uh of class, it's which is the last week uh of class, it…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 10: 19 to actually present your poster in class you know tell your classmate what you you know tell your classmate what you you know tell your classmate what you did and then the third thing is you did and t…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 10: 58 right. Uh so a couple of things to remind you guys is that uh this class is remind you guys is that uh this class is remind you guys is that uh this class is completely AI friendly and open completely…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 11: 53 people. However, uh because we're so people. However, uh because we're so flexible on our you know just like open flexible on our you know just like open flexible on our you know just like open source…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 12: 33 like it's totally fine and yeah just in like it's totally fine and yeah just in general do not plagiarize. It's you know general do not plagiarize. It's you know general do not plagiarize. It's you kn…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 13: 20 basically um so uh myself and the TAs are going to so uh myself and the TAs are going to so uh myself and the TAs are going to start hosting office hours um next week. start hosting office hours um ne…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 14: 00 they do know um you know they they have done research in terms of well Douglas done research in terms of well Douglas done research in terms of well Douglas has done research in terms of flow has done…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 14: 44 probably your classmate are going to figure like a lot of things out on your figure like a lot of things out on your figure like a lot of things out on your own through ex own through ex own through e…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 15: 27 by anything I mean you're not allowed to by anything I mean you're not allowed to use your laptop at all. So just going to use your laptop at all. So just going to use your laptop at all. So just goin…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 16: 07 website. Basically, they're just like website. Basically, they're just like easy conceptual questions. Uh, and they easy conceptual questions. Uh, and they easy conceptual questions. Uh, and they shou…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 16: 54 with me uh to collect feedback on your learning experience in this class. And learning experience in this class. And learning experience in this class. And uh yeah basically it's just going to be uh y…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 17: 31 talk about like how people have been trying to improve upon those uh basic trying to improve upon those uh basic trying to improve upon those uh basic algorithms throughout the years. Uh and algorithm…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 18: 17 then week six is going to be we're going then week six is going to be we're going to be talking about probably a lot of to be talking about probably a lot of to be talking about probably a lot of you …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 19: 06 post it on there. Um and uh post it on there. Um and uh the gray scope, which I have not add the gray scope, which I have not add the gray scope, which I have not add anyone to, by the way, so don't p…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 19: 56 have announcement thank you um forum uh have announcement thank you um forum uh uh uh discussions you and feel free to uh uh discussions you and feel free to uh uh discussions you and feel free to sha…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 20: 39 constant basis? Raise your hand if you do. All right. Amazing. All right. Amazing. Actually uh a lot of people do. For Actually uh a lot of people do. For Actually uh a lot of people do. For those of …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 21: 30 free. Uh so in total you should be able to get around like 280 GP hours on L4DS. to get around like 280 GP hours on L4DS. to get around like 280 GP hours on L4DS. Um and that should be plenty and shou…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 22: 19 AWS is, but it's just like another uh AWS is, but it's just like another uh cloud comput service that is very very cloud comput service that is very very cloud comput service that is very very popular…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 23: 05 you are a MLDD student or LTI student or probably any SCES student at this point probably any SCES student at this point probably any SCES student at this point uh you can probably get access to the u…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 23: 45 any lab clusters, uh then uh I have listed a bunch of uh freeish and listed a bunch of uh freeish and listed a bunch of uh freeish and cheapish uh cloud comput service that cheapish uh cloud comput se…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 24: 40 you're not using it. Do not go bankrupt you're not using it. Do not go bankrupt uh when you're training models. This is uh when you're training models. This is uh when you're training models. This is …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 25: 23 finalize the list of registered student and uh yeah but like whether or not you and uh yeah but like whether or not you and uh yeah but like whether or not you are going to be able to register please …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 26: 02 And uh oh yeah, one other thing that I need to mention there are a couple of need to mention there are a couple of need to mention there are a couple of people already done that but which you people a…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 26: 43 streamed on Zoom like we have seen here. Thank you Jacob for doing that. Um so Thank you Jacob for doing that. Um so Thank you Jacob for doing that. Um so yeah so basically what you can do here yeah s…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 27: 27 Ponupto I don't know uh like the the the Ponupto I don't know uh like the the the thing that I also shared on Discord and thing that I also shared on Discord and thing that I also shared on Discord an…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 28: 09 uh feel free to read it especially this one feel free to read it especially this one feel free to read it especially this one book uh so this book is written by a book uh so this book is written by a …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 28: 50 going to be the same um yeah so feel free to read this book this is amazing free to read this book this is amazing free to read this book this is amazing uh and besides this class if you want to uh an…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 29: 37 and diffusion and flow matching models. and diffusion and flow matching models. Uh so nothing is important, nothing else Uh so nothing is important, nothing else Uh so nothing is important, nothing el…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 30: 14 in the class if you don't want to. Uh I in the class if you don't want to. Uh I know I don't like taking classes uh when know I don't like taking classes uh when know I don't like taking classes uh wh…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 30: 51 some of my survey you know if you need some of my survey you know if you need some extra credits. Um but anyway um some extra credits. Um but anyway um some extra credits. Um but anyway um yeah but ju…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 32: 01 given Y. Is that what you said? Okay. So, but what is P? >> I mean, I guess I could model X. >> I mean, I guess I could model X. >> Okay. Yeah. Yeah. Yeah. Pretty much. >> Okay. Yeah. Yeah. Yeah. Pret…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 33: 04 of Y right what are they? So X and Y's of Y right what are they? So X and Y's are random variables. So basically just are random variables. So basically just are random variables. So basically just th…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 33: 47 know which one is more unfortunate actually. Um but uh basically we usually actually. Um but uh basically we usually actually. Um but uh basically we usually denote it as P of X. denote it as P of X. …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 34: 38 consider X or only consider Y. And the consider X or only consider Y. And the conditional distribution is when you are conditional distribution is when you are conditional distribution is when you are…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 35: 21 just do a little bit of algebra you're just do a little bit of algebra you're going to be able to get this very nice going to be able to get this very nice going to be able to get this very nice formu…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 36: 11 y tells you nothing about x or knowing x tells you nothing about y. So you uh tells you nothing about y. So you uh tells you nothing about y. So you uh just do not change your posterior. Oh just do no…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 37: 01 person distribution we have like the person distribution we have like the average rate as our uh parameter. average rate as our uh parameter. average rate as our uh parameter. Uh basically it's just l…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 37: 57 we call the probability of the data we call the probability of the data given our learn the model parameter as given our learn the model parameter as given our learn the model parameter as the likelih…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 39: 28 Richard Feman. Uh apparently people found it on his blackboard after he died found it on his blackboard after he died found it on his blackboard after he died or something. I don't know if that after …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 40: 06 bunch of images of a bedroom uh and then the label of the bedroom is whether or the label of the bedroom is whether or the label of the bedroom is whether or not it's a luxurious bedroom. I've been no…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 40: 51 we're trying to learn is what a bedroom should look like or what a luxurious or should look like or what a luxurious or should look like or what a luxurious or what a cheap I guess uh bedroom should w…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 41: 36 should be? So basically do we believe this is a gausian? Do we believe this is this is a gausian? Do we believe this is this is a gausian? Do we believe this is a mixture of gausian? Uh what kind of a…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 42: 26 necessarily able to uh be able to tell necessarily able to uh be able to tell like how likely are your uh like is some like how likely are your uh like is some like how likely are your uh like is some…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 43: 06 labeling or like uh annotations and stuff like that. Okay, stuff like that. Okay, stuff like that. Okay, cool. So how to train your generated cool. So how to train your generated cool. So how to train…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 44: 05 if my um parameters are correct then the if my um parameters are correct then the real data point should have high real data point should have high real data point should have high likelihood. Uh whic…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 44: 52 image is composed by a bunch of um pixels pixels pixels uh then um by the probability chain rule uh then um by the probability chain rule uh then um by the probability chain rule which just means that…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 45: 39 able to get this thing that is very nice where the log likely the conditional log where the log likely the conditional log where the log likely the conditional log likelihood of your case element is o…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 46: 23 LLMs. Yes, we got LM already. Yeah, LLMs. Yes, we got LM already. Yeah, super easy. This is why people love it. super easy. This is why people love it. super easy. This is why people love it. Um, yeah…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 47: 07 we cannot see directly. Uh so for example how a person look is largely example how a person look is largely example how a person look is largely determined by their genes which is not determined by th…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 47: 52 such a beautiful paper. I was like oh my god this this thing is like who who god this this thing is like who who god this this thing is like who who thought of this thing? This is like thought of this…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 48: 37 maximizing the likelihood of the data we maximizing the likelihood of the data we also need to do another thing which is also need to do another thing which is also need to do another thing which is t…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 49: 19 two distributions. So we're trying to two distributions. So we're trying to minimize the difference between the minimize the difference between the minimize the difference between the distribution of …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 50: 04 measure it by like so all the divergence that you output should be greater than that you output should be greater than that you output should be greater than zero and it should only be equal greater z…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 50: 53 kind of the intuition. Uh but basically what's going to happen here is that what's going to happen here is that what's going to happen here is that we're going to be using the KL we're going to be usi…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 51: 41 when you're trying to decode it given when you're trying to decode it given unz and then the other thing is what we unz and then the other thing is what we unz and then the other thing is what we call…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 52: 29 a expectation of this thing and then like By bas rule again you are going to like By bas rule again you are going to like By bas rule again you are going to be able to get from the log px is be able t…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 53: 16 part is the the elbow the elbow. The second part is the kale the elbow. The second part is the kale the elbow. The second part is the kale divergence that we're trying to uh divergence that we're tryi…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 54: 07 distribution uh like then it's just distribution uh like then it's just going to be like the the the the going to be like the the the the going to be like the the the the integral of the the joint uh …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 55: 03 >> yeah because the because log is concave. Yeah, that's right. Uh so yeah, Jensen Yeah, that's right. Uh so yeah, Jensen Yeah, that's right. Uh so yeah, Jensen inequality here and we get an inequalit…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 55: 51 about today. It's called uh Catch Me If about today. It's called uh Catch Me If You Can. Uh basically, I don't know if You Can. Uh basically, I don't know if You Can. Uh basically, I don't know if you…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 56: 27 he was wearing a suit or something like he was wearing a suit or something like that and people were laughing me hah and that and people were laughing me hah and that and people were laughing me hah a…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 57: 08 but because he's getting like really good at like faking things so he's able good at like faking things so he's able good at like faking things so he's able to like fool the detective in into to like …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 57: 45 uh the model that we're going to be talking about. So you have a generator talking about. So you have a generator talking about. So you have a generator uh and then you have a discriminator. uh and th…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 58: 28 network or GAN. Uh this was also developed in I guess 2014. So you see developed in I guess 2014. So you see developed in I guess 2014. So you see how like a decade ago I don't know how like a decade …
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 59: 02 first just like sample something um from a easy to sample distribution such as a easy to sample distribution such as a easy to sample distribution such as like gausian and then you have a like gausian…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 59: 48 that uh basically this is just a regular you know classification loss right like you know classification loss right like you know classification loss right like binary classification law for the for b…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 60: 59 something from it I guess why does the well actually this is not exactly the well actually this is not exactly the well actually this is not exactly the case we're going to be talking about how case w…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 61: 37 we don't even have P of Z given X in our final loss. Actually our final loss is final loss. Actually our final loss is final loss. Actually our final loss is this thing. So our final loss is this this…
cmu-10799-diffusion-flow L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling 62: 30 understand how diffusion works. Uh so in understand how diffusion works. Uh so in uh general next class we're going to be uh general next class we're going to be uh general next class we're going to b…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 00: 03 To really understand about how this To really understand about how this video links to image diffusion, I video links to image diffusion, I video links to image diffusion, I believe that we first need…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 00: 49 two. So this is my vector. That's how you represent this vector. That's how you represent this vector. That's how you represent this vector. But how would you represent an entire But how would you rep…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 01: 31 this is 128x 128 pixels, I will arrange these 128x 128 pixels in a in a these 128x 128 pixels in a in a these 128x 128 pixels in a in a u column. So imagine that you have this u column. So imagine tha…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 02: 23 somewhere here in a pixel space. This is another image. It may live somewhere another image. It may live somewhere another image. It may live somewhere here in pixel space. So pixel space is here in p…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 03: 17 you really need to understand. How is the concept of probability How is the concept of probability How is the concept of probability distribution linked to that of an image distribution linked to that…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 04: 03 and this is what I would have explained to you if chat GPT or Gemini did not to you if chat GPT or Gemini did not to you if chat GPT or Gemini did not exist but now let me actually take you exist but …
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 06: 01 Let's see. So the reason I'm showing Let's see. So the reason I'm showing this to you is because I want this this to you is because I want this this to you is because I want this learning material to …
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 06: 46 right? It wouldn't even look like a cat or a tree. It would look like random or a tree. It would look like random or a tree. It would look like random noise. So 99% of points in the pixel noise. So 99…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 07: 43 where all Chinese symbols are living. where all Chinese symbols are living. That's the probability distribution of That's the probability distribution of That's the probability distribution of this im…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 08: 29 higher probability of finding the Chinese symbol. Right? So now if you Chinese symbol. Right? So now if you Chinese symbol. Right? So now if you take a look at this 3D plot, right? If I take a look at…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 09: 19 pixels. Pixels are converted into flat vectors. Those are represented in higher vectors. Those are represented in higher vectors. Those are represented in higher dimensional space. dimensional space. …
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 10: 13 saying that we are finding this this probability distribution. If you ask if probability distribution. If you ask if probability distribution. If you ask if you go to chat GPT right now or let's you g…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 11: 26 finding the true probability distribution or getting as close to it distribution or getting as close to it distribution or getting as close to it as possible. Right? So as possible. Right? So as possi…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 12: 25 Right? So if I the red line is my true Right? So if I the red line is my true probability distribution probability distribution probability distribution and the purple line the purple line is and the …
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 13: 22 definitely do look like Chinese symbols definitely do look like Chinese symbols because this purple probability because this purple probability because this purple probability distribution is close to…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 14: 16 sampled that purple probability distribution. distribution. Sampled from that purple probability Sampled from that purple probability Sampled from that purple probability distribution and created this…
diffusion-lm-vizuara L4: Lecture 3: Generative AI through a probabilistic lens 15: 26 underlying probability distribution. Let me write it here. Purpose Purpose of a diffusion model is to find Purpose of a diffusion model is to find the underlying probability distribution. That is the …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 00: 03 Hello everyone and welcome to the first Hello everyone and welcome to the first lecture of the course principles of lecture of the course principles of lecture of the course principles of diffusion mo…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 01: 40 the heights of maybe a few friends of yours and uh you have noted them down in yours and uh you have noted them down in yours and uh you have noted them down in your diary. your diary. your diary. So …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 02: 47 your school and uh you want to make a your school and uh you want to make a guess a reasonable guess about what guess a reasonable guess about what guess a reasonable guess about what would be the hei…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 03: 50 will be that I think the new student will have uh you know a height of maybe will have uh you know a height of maybe will have uh you know a height of maybe 5' 5 in. 5' 5 in. 5' 5 in. So your predicti…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 05: 15 are very less to be seen. Now if we have this distribution with Now if we have this distribution with Now if we have this distribution with us, we can easily say that us, we can easily say that us, we…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 06: 29 curve is the true distribution. So this is an example of a decent match. So this is an example of a decent match. So this is an example of a decent match. I can see that the model distribution I can s…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 07: 24 height of the new student what I see is that I'll make a guess that okay fine that I'll make a guess that okay fine that I'll make a guess that okay fine the new student will have a height of the new …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 08: 24 Uh true distribution. So true distribution is something which is uh in distribution is something which is uh in distribution is something which is uh in in supervised learning you're trying to in supe…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 09: 34 models are trained to do. They are trained so that the predicted trained so that the predicted trained so that the predicted distribution matches as close as distribution matches as close as distribut…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 10: 35 see something visually it stays in our mind much longer than reading a text or mind much longer than reading a text or mind much longer than reading a text or hearing from someone. So let's look at he…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 12: 11 So visually if you just keep this So visually if you just keep this uh video in mind and uh video in mind and uh video in mind and you saw how the initial curve slowly you saw how the initial curve sl…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 13: 37 probably not thought of it as a case probably not thought of it as a case study which comes under the domain of study which comes under the domain of study which comes under the domain of deep generat…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 15: 00 and the deep generative model outputs a trained neural network that distribution. So there are three main points here. The So there are three main points here. The So there are three main points here.…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 16: 03 So there are two main goals of deep generative modeling. generative modeling. generative modeling. The first goal is that we always aim for The first goal is that we always aim for The first goal is t…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 17: 04 generative modeling. Realistic generation and controllable generation. generation and controllable generation. generation and controllable generation. Now let's move to the goal of deep Now let's move…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 18: 09 named as generative AI. But where does the name generative really come from? It the name generative really come from? It the name generative really come from? It it comes from the development of it co…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 19: 10 to generate new images of cats in this to generate new images of cats in this case. Okay. So now let's formally look at a Okay. So now let's formally look at a more complex example more complex exampl…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 20: 21 gives gives that to us beforehand. We get a small sneak peek into the true We get a small sneak peek into the true We get a small sneak peek into the true distribution through the data samples distrib…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 21: 24 represent a probability distribution. So orange contour is the true So orange contour is the true So orange contour is the true probability distribution and the black probability distribution and the …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 22: 33 they developed they developed models which are trained to approximate models which are trained to approximate models which are trained to approximate the true data distribution of cats and the true da…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 23: 30 whenever we look at a cat in our mind we know that this is a cat because we have know that this is a cat because we have know that this is a cat because we have formed some assumption of the formed so…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 24: 36 if we take two numbers I can clearly see that the the discrepancy is the that the the discrepancy is the that the the discrepancy is the difference between the two numbers. difference between the two …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 25: 38 screen. I'm initially going to just run this video and then um I'm going to this video and then um I'm going to this video and then um I'm going to explain different parts of it. Okay. So let's try to…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 28: 10 difference is a great start but it doesn't capture everything. doesn't capture everything. doesn't capture everything. For example, if we take the absolute For example, if we take the absolute For exa…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 29: 13 between the true distribution and the model distribution is actually less than model distribution is actually less than model distribution is actually less than one. which means that our model has one…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 30: 13 So we take the log of the ratio for all So we take the log of the ratio for all these different points and then just add these different points and then just add these different points and then just a…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 31: 16 into account by multiplying by a factor which is dependent on how likely the which is dependent on how likely the which is dependent on how likely the case is under the true distribution. case is unde…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 32: 24 distribution with a model distribution Q. It is zero when P and Q are identical Q. It is zero when P and Q are identical Q. It is zero when P and Q are identical and grows as Q becomes worse. Especial…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 33: 33 how do we start to train these deep generative models. So let's understand generative models. So let's understand generative models. So let's understand that. objectives of objectives of deep generati…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 35: 07 should always add up to one. should always add up to one. Take an example of a coin toss. Take an example of a coin toss. Take an example of a coin toss. Probability of heads is 0.5. Probability Proba…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 36: 05 of these operations will make sure that of these operations will make sure that the output is positive. A very common the output is positive. A very common the output is positive. A very common choice…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 37: 04 you divide all of these numbers by 90. So you get Alice, Bob and Charlie as 44, So you get Alice, Bob and Charlie as 44, So you get Alice, Bob and Charlie as 44, 33 and 22. And these numbers add up 33…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 38: 10 finding the scores for every single sample in this contour is going to be sample in this contour is going to be sample in this contour is going to be challenging because we do not even know challengin…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 39: 11 uh we are now going to briefly discuss uh we are now going to briefly discuss the different families of deep the different families of deep the different families of deep generative models. generative…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 40: 17 more probable and higher energy to data points which are less probable. points which are less probable. points which are less probable. So imagine that we collect the exam So imagine that we collect t…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 41: 17 Now what happens is that we want a model that can understand this we want a model that can understand this we want a model that can understand this pattern. So energy based models do not pattern. So e…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 42: 15 numbers around 80 have a lower energy. numbers around 60 have a lower energy. numbers around 60 have a lower energy. numbers around 60 have a lower energy. Numbers around 30 which are not very Numbers…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 43: 12 should receive a higher score. Once we understand this simple rule, the understand this simple rule, the understand this simple rule, the objective of energy based models is to objective of energy bas…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 44: 48 So energy based models uh was the first starting point for deep was the first starting point for deep was the first starting point for deep generative modeling. generative modeling. Then we moved on t…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 46: 02 After this context is seen by the LLM, After this context is seen by the LLM, it predicts the next token it predicts the next token it predicts the next token based on the token which gets the based o…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 47: 01 know take a look at this video first know take a look at this video first which explains variational autoenccoders which explains variational autoenccoders which explains variational autoenccoders at …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 47: 59 image flowing through a autoenccoder you can see that the latin space the image can see that the latin space the image can see that the latin space the image is blurred because we are reducing the is …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 48: 54 which is devoted exactly on this. The next prominent milestone is called as next prominent milestone is called as next prominent milestone is called as normalizing flows. normalizing flows. normalizin…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 50: 01 some simple distribution and you twist it, bend it, turn it to reach the true it, bend it, turn it to reach the true it, bend it, turn it to reach the true distribution which is exactly what the distr…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 51: 24 investigate investigate the difference between generated art and the difference between generated art and the difference between generated art and deep fix. We basically have to identify We basically …
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 52: 45 there is a generator and a discriminator. The generator discriminator. The generator discriminator. The generator generates images and the role of the generates images and the role of the generates im…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 54: 19 not be able to recognize this as a fake not be able to recognize this as a fake image. image. image. So see here real image detected the the So see here real image detected the the So see here real im…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 55: 15 student heights. But we only had a few student heights. But we only had a few data samples. So we had to use those data samples. So we had to use those data samples. So we had to use those data sample…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 56: 10 distribution mainly we looked at finding the normalizing factor is very difficult the normalizing factor is very difficult the normalizing factor is very difficult and uh that's why we need more and u…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 57: 04 families uh illustrate complimentary strategies for modeling complex strategies for modeling complex strategies for modeling complex distribution and they also provide distribution and they also provi…
diffusion-principles-vizuara L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models 57: 56 look at very practical examples as as we move along. move along. move along. Thank you very much everyone and uh I Thank you very much everyone and uh I Thank you very much everyone and uh I hope this…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 00: 05 Hello everyone and welcome to the second Hello everyone and welcome to the second lecture of the course principles of lecture of the course principles of lecture of the course principles of diffusion …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 01: 17 from scratch. from scratch. And this lecture is And this lecture is And this lecture is going to be very important for the going to be very important for the going to be very important for the subsequ…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 02: 19 with the understanding of VAS. with the understanding of VAS. So first we'll take a simple example. So first we'll take a simple example. So first we'll take a simple example. Imagine that you have co…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 03: 19 in cursive. Some of us u don't write in cursive. So there are a u don't write in cursive. So there are a u don't write in cursive. So there are a lot of different ways in which people lot of different…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 04: 26 Then another time you press a button. Another time you get another hello. This Another time you get another hello. This Another time you get another hello. This time it might be some different student…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 05: 35 such that or generate a machine such that the samples drawn from that machine that the samples drawn from that machine that the samples drawn from that machine are from the probability distribution ar…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 06: 45 factors that determine the style of the factors that determine the style of the handwriting. handwriting. handwriting. So each student handwriting depends on So each student handwriting depends on So …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 08: 03 So my first intuition is that okay fine So my first intuition is that okay fine my first job is to figure out all these my first job is to figure out all these my first job is to figure out all these …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 09: 02 final shape of the handwriting. final shape of the handwriting. So once you capture the secret recipe So once you capture the secret recipe So once you capture the secret recipe for for for a student …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 09: 57 machine looks as follows. machine looks as follows. Secret recipe is given as an input to Secret recipe is given as an input to Secret recipe is given as an input to the machine. The machine takes thi…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 10: 52 interesting, right? It it it looks like okay fine this this it it makes a lot of okay fine this this it it makes a lot of okay fine this this it it makes a lot of sense. [snorts] sense. [snorts] sense…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 11: 53 now um in the rest of this lecture we We now um in the rest of this lecture we We are going to assume that are going to assume that are going to assume that for the handwriting example that we took fo…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 12: 54 not know these factors beforehand. That is why they are called as latent is why they are called as latent is why they are called as latent variables. variables. variables. So So intuitively we think t…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 13: 51 y-axis is represented by the symbol zed 2. Zed1 stands for slantness of the 2. Zed1 stands for slantness of the 2. Zed1 stands for slantness of the handwriting and zed2 stands for neatness handwriting…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 14: 48 below it's it's not neat. So I have made some spelling mistake So I have made some spelling mistake So I have made some spelling mistake here. But what I want to show is I want here. But what I want t…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 15: 33 mapping which I was talking about. For mapping which I was talking about. For example, if you pick a point here, you example, if you pick a point here, you example, if you pick a point here, you know …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 16: 31 in your class. in your class. So this is also called as the So this is also called as the So this is also called as the distribution in the latent space of the distribution in the latent space of the …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 17: 37 typical handwriting typical handwriting let's say it is divided into 28x 28 let's say it is divided into 28x 28 let's say it is divided into 28x 28 pixels. pixels. pixels. So there are 784 numbers rig…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 18: 39 latent space is has a mean of zero and a latent space is has a mean of zero and a standard deviation of one or any other standard deviation of one or any other standard deviation of one or any other v…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 19: 27 So for now what we have seen is that we have this general architecture for the have this general architecture for the have this general architecture for the machine which takes the secret recipe as ma…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 20: 34 another word for this machine which is called as decoder. called as decoder. called as decoder. Let us quickly summarize what all we Let us quickly summarize what all we Let us quickly summarize what …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 21: 44 vary. So here you can see that as the yellow dot moves around the Latin space, yellow dot moves around the Latin space, yellow dot moves around the Latin space, the handwriting on the right hand side …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 22: 44 next. So far we have used the decoder to generate samples from the latent generate samples from the latent generate samples from the latent variables. But what is this decoder variables. But what is t…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 23: 55 we create a latent space with the latent variables. So just as before we assume variables. So just as before we assume variables. So just as before we assume that there are two latent variables zed th…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 24: 54 know. [snorts] Now let's say we have chosen a point in Now let's say we have chosen a point in Now let's say we have chosen a point in the Latin space which corresponds to the the Latin space which co…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 25: 56 we can represent digit 5 as a bunch of we can represent digit 5 as a bunch of pixels pixels pixels which are divided as 28 into 28 pixels where some pixels have a value of where some pixels have a val…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 27: 08 five, if you closely look at it, it it does look like a five. It's it's it's does look like a five. It's it's it's does look like a five. It's it's it's almost something like this. Okay. So it looks l…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 28: 08 is that with this approach we will get a fixed shape of five every time but we fixed shape of five every time but we fixed shape of five every time but we will not get variations of it and what will n…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 29: 10 restricted to zero and one. You are open to the possibility of to the possibility of to the possibility of a color which appears in between white a color which appears in between white a color which a…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 30: 08 the decoder does not give a single value but for every pixel the decoder gives but for every pixel the decoder gives but for every pixel the decoder gives the mean the mean the mean and the standard d…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 31: 13 deterministic approach only. So now let us So now let us So now let us have a simple example to understand how have a simple example to understand how have a simple example to understand how does the …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 32: 16 So the decoder is nothing but a neural network which is trained to take inputs network which is trained to take inputs network which is trained to take inputs from the latin space and generate from th…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 33: 11 somewhere towards the right, which means that it has to be a bit bright and the that it has to be a bit bright and the that it has to be a bit bright and the pixel value is high. Bright pixels have pi…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 34: 10 now we have covered now we have covered one part of the story which explains the one part of the story which explains the one part of the story which explains the variational autoenccoder. variational…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 35: 10 part, you will understand that we have made a major assumption. made a major assumption. made a major assumption. Remember when we talked about the Remember when we talked about the Remember when we t…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 36: 14 recipe and then you need to map all these handwriting styles to the secret these handwriting styles to the secret these handwriting styles to the secret recipe and make all those points and recipe and…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 37: 13 the latin space to access to generate the latin space to access to generate digit five. One option is to access all digit five. One option is to access all digit five. One option is to access all poss…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 38: 04 Wouldn't it be better if we knew which part of the Latin space to access for part of the Latin space to access for part of the Latin space to access for the type of image we want to generate? the type…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 39: 00 decoder. So the overall architecture looks like So the overall architecture looks like So the overall architecture looks like this. this. this. First I get let's say you go to any First I get let's sa…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 40: 00 this architecture is very similar to a plain autoenccoder where you have an plain autoenccoder where you have an plain autoenccoder where you have an encoder which compresses the image into encoder wh…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 41: 03 a dis or or a region which is the most a dis or or a region which is the most probable region of finding the digit probable region of finding the digit probable region of finding the digit five instea…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 42: 01 we are going to look at a nice visual we are going to look at a nice visual description to understand this much description to understand this much description to understand this much better. better. …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 43: 12 individual points. So 99% of these points, they they produce garbage. So points, they they produce garbage. So points, they they produce garbage. So your your space your latent space is not your your …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 44: 14 regions which tell you that this is the region corresponding to the dog. This is region corresponding to the dog. This is region corresponding to the dog. This is the region corresponding to the cat e…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 45: 08 space which is continuous and the space which is continuous and the garbage gaps are now filled with garbage gaps are now filled with garbage gaps are now filled with distributions. So now you can saf…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 46: 07 So the two stories put together form the variational autoenccoder. variational autoenccoder. We first looked at the decoder where we We first looked at the decoder where we We first looked at the deco…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 47: 03 the areas in the latin space? How do you know which levers to pull in the know which levers to pull in the know which levers to pull in the decoder? This is given to you by the decoder? This is given …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 48: 08 uh styles of the students when they uh styles of the students when they wrote the word hello or the handwritten wrote the word hello or the handwritten wrote the word hello or the handwritten digits a…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 49: 09 going to give you the latent space distribution given the image. So that's distribution given the image. So that's distribution given the image. So that's why you see the notation kind of flipped why …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 50: 09 the encoder like this and the decoder the encoder like this and the decoder going from a smaller line here to a going from a smaller line here to a going from a smaller line here to a bigger line. In …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 51: 16 these variational autoenccoders trained. How do you train these to generate How do you train these to generate How do you train these to generate images? Let's say you give a sample of images? Let's s…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 52: 05 and then sample from it. So we are going to take a real example in the next class to take a real example in the next class to take a real example in the next class where you will see exactly how deep …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 52: 56 is not deterministic but rather it is is not deterministic but rather it is probabilistic. probabilistic. probabilistic. And finally in your mind I want you to And finally in your mind I want you to A…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 53: 48 image which is generated at the end. This is all that is happening in the This is all that is happening in the This is all that is happening in the variation autoenccoders. variation autoenccoders. va…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 55: 04 tells you the hidden factors of variation behind that image. variation behind that image. variation behind that image. [snorts] The machine tells you the hidden factors The machine tells you the hidde…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 56: 10 can lie. So it appears to be a very can lie. So it appears to be a very smart and a clever machine. smart and a clever machine. smart and a clever machine. Okay. Now you have these hidden factors Okay…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 57: 11 the final image should be as close as possible to the original image. So you possible to the original image. So you possible to the original image. So you can see that there has to be some kind can se…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 58: 10 encode and the second neural network is the machine which learns to decode. the machine which learns to decode. the machine which learns to decode. So somehow we have to train the So somehow we have t…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 59: 20 a dot like this and you pass it to the a dot like this and you pass it to the decoder and you get a value of zero decoder and you get a value of zero decoder and you get a value of zero which which me…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 60: 07 our predicted distribution which should match as close as possible to the true match as close as possible to the true match as close as possible to the true distribution which we do not really distrib…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 61: 19 points in the latin space which is also given in this formula. given in this formula. given in this formula. I I don't want to stress too much on how I I don't want to stress too much on how I I don't…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 62: 21 the visualization before where we saw that it's like finding or that it's like finding or that it's like finding or sampling for from a very giant ocean and sampling for from a very giant ocean and sa…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 63: 17 that point being real? And this actually does not even make use And this actually does not even make use And this actually does not even make use of the encoder. This is all decoder of the encoder. Th…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 64: 26 Now the question is that what is the Now the question is that what is the elbow objective? [snorts] elbow objective? [snorts] elbow objective? [snorts] Well to understand this let's understand Well to…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 65: 25 I will not get very mathematical here and I will not focus too much on the and I will not focus too much on the and I will not focus too much on the derivation but I will provide a link in derivation …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 66: 24 So this essentially says that whatever image you're looking at the end whatever image you're looking at the end whatever image you're looking at the end is that image is that image is that image uh as…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 67: 33 in any fashion but in more realistic settings the in more realistic settings the in more realistic settings the distribution is more likely to be distribution is more likely to be distribution is more…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 68: 30 will be average in terms of the will be average in terms of the slantness, in terms of the neatness etc. slantness, in terms of the neatness etc. slantness, in terms of the neatness etc. So we are So …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 69: 32 how likely is the reconstructed um output. um output. um output. What is the probability of that? And we What is the probability of that? And we What is the probability of that? And we want to maximiz…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 70: 35 our encoder distribution to stay as close as possible to a gshian which close as possible to a gshian which close as possible to a gshian which comes from our human intuition that comes from our human…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 71: 38 want to compress the earth cat into something which captures the essence of something which captures the essence of something which captures the essence of the cat. We want to send a secret recipe the…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 72: 30 objective and we try to maximize the elbow. elbow. elbow. So since the true objective is always So since the true objective is always So since the true objective is always greater than the elbow, maxi…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 73: 27 want to move our code to the yellow distribution. We want to move it as distribution. We want to move it as distribution. We want to move it as close as possible to the normal close as possible to the…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 74: 23 what remains and it makes sense from an what remains and it makes sense from an intuitive standpoint because later intuitive standpoint because later intuitive standpoint because later people forget t…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 75: 20 going to come together very nicely in this last practical example. Our task is this last practical example. Our task is this last practical example. Our task is to train a variation autoenccoder to to…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 76: 21 hidden factors which we have absolutely hidden factors which we have absolutely no clue what they are but the encoder is no clue what they are but the encoder is no clue what they are but the encoder …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 77: 13 So remember our decoder setup looks like this. It's it it needs to take the this. It's it it needs to take the this. It's it it needs to take the sample from the latent space and then it sample from t…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 78: 10 So it should take the latent vector and So it should take the latent vector and it should generate the reconstructed it should generate the reconstructed it should generate the reconstructed image. Ri…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 79: 07 first machine do? The first machine actually does the reverse. It takes the actually does the reverse. It takes the actually does the reverse. It takes the image as the input, the real image, and imag…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 80: 13 region which has the mu of.5 comma.5 and region which has the mu of.5 comma.5 and the sigma of 2a 2. Right? So you need the sigma of 2a 2. Right? So you need the sigma of 2a 2. Right? So you need two …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 81: 01 the GitHub repo in the description the GitHub repo in the description section. section. section. [snorts] [snorts] Okay. So overall the encoder decoder Okay. So overall the encoder decoder Okay. So ov…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 81: 56 So this is our entire encoder decoder So this is our entire encoder decoder architecture whatever we discussed as architecture whatever we discussed as architecture whatever we discussed as this machi…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 82: 51 This compares every pixel of the input with the output. If the original pixel with the output. If the original pixel with the output. If the original pixel is black and the VA predicts white, the is b…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 83: 46 zero and a standard deviation of one. To ensure that the mean is zero, we add a ensure that the mean is zero, we add a ensure that the mean is zero, we add a penalty if the mean deviates from zero. pe…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 84: 40 because log of a small value is a big uh number and there is a negative sign number and there is a negative sign number and there is a negative sign associated with it. So this will always associated …
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 85: 34 the uh description. But first I just want to want to want to describe everything that we have describe everything that we have describe everything that we have conceptually described in a mathematical…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 86: 26 for longer duration maybe we'll get an for longer duration maybe we'll get an even better distribution which is even better distribution which is even better distribution which is centered at zero and…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 87: 21 this training proceeds and this is the this training proceeds and this is the quality of the reconstructions because quality of the reconstructions because quality of the reconstructions because we ha…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 88: 15 latin space uh you see that the it's it's kind of blur in some cases which it's kind of blur in some cases which it's kind of blur in some cases which we'll come to in in in some time but you we'll co…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 89: 05 is that it it produces blurry outputs and there is a very nice reason for it. and there is a very nice reason for it. and there is a very nice reason for it. I will share the link in the description I…
diffusion-principles-vizuara L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models 89: 57 applications I have seen and uh there are encoders and distribute and and are encoders and distribute and and are encoders and distribute and and decoders right so you also have decoders right so you …