Extracting and Composing Robust Features with Denoising Autoencoders

What is a Denoising Autoencoder?


The Denoising Autoencoder is a modification to the basic autoencoder.
The denoising autoencoder partially corrupts the initial input vector x\mathbf{x} in a stochastic manner:
x~(i)MD(x~(i)x(i))\tilde{\mathbf{x}}^{(i)} \sim \mathcal{M}_\mathcal{D}(\tilde{\mathbf{x}}^{(i)} \vert \mathbf{x}^{(i)})
The model is then trained to recover the original input by minimizing:
LDAE(θ,ϕ)=1ni=1n(x(i)fθ(gϕ(x~(i))))2L_\text{DAE}(\theta, \phi) = \frac{1}{n} \sum_{i=1}^n (\mathbf{x}^{(i)} - f_\theta(g_\phi(\tilde{\mathbf{x}}^{(i)})))^2
where MD\mathcal{M}_\mathcal{D} defines the mapping from the true data samples to the noisy or corrupted ones.

Machine Learning Research Flashcards is a collection of flashcards associated with scientific research papers in the field of machine learning. Best used with Anki or Obsidian. Edit MLRF on GitHub.